RFC: what's in a stable series?

RFC: what's in a stable series?

[Jeff Garzik]

So, I was wondering if it is possible to find a consensus on what sorts 
of changes are ok in a stable series, as it matures.  This mainly 
applies to 2.4 right now, but I am interested in thoughts on 2.6.x also.


The recent "big fuss over a small change", the direct_IO API change, is 
indicative to me of a larger question:  what sort of API changes are 
acceptable in a stable series?

Linux has traditionally been fairly loose with caring about API changes. 
  We "try" to make sure major changes all occur during the development 
series, but reality doesn't always play out so nicely.  The 
oft-mentioned chicken-n-egg situation:  people really only test the 
stable series' heavily, so problems that should have been caught during 
development (if the world was perfect) are instead caught during the 
stable series.  New hardware and features appear, that our existing APIs 
do not take into account.

First example is the direct_IO change.  There are valid reasons to 
include the change, and valid reasons to avoid it.  Vendors (and 
-ac/-aa?) appear to be shipping this non-standard direct_IO API.  It's a 
good change and has definite value, so why not include it in the 2.4.x 
kernel?  The other side of the argument is breakage not immediately 
visible:  the people maintaining drivers that support multiple kernels 
must deal with N variations of the same API, within the stable series.

I know the gut reaction on lkml is often to trash these people, but 
let's be realistic.  Linux is huge these days.  Drivers normally start 
life outside the tree, and then once acceptable, are merged.  Other 
drivers must be cross-kernel compatible in order to support all the 
customers that the hardware vendor decides to support.  Red Hat, SuSE, 
Mandrake, etc. distros in the field are not automagically upgraded -- so 
if a driver for those existing releases is desired, one must deal with 
all the API changes.  API changes in a stable series instantly make life 
harder on all these people.

The flip side of the coin is progress.

If most vendors are shipping a non-mainstream change, why not make it 
mainstream?
Linux users buying new, cool hardware would logically want a driver from 
the stable series to drive their hardware.  If API changes are required 
to support this hardware, why not go ahead and make the change?  It will 
increase Linux acceptable, market penetration, make people happy, etc., etc.

Adding into the mix, until recently, the 2.4.x series was proceeding at 
a glacial pace.  I think Marcelo has demonstrated that this situation is 
changes, but we still need to consider it's effects.  2.4.21 was a 
fairly large chunk of changes, because it took so long.  2.4.22 is 
absorbing the after-effects of those changes.  So one would expect that, 
(a) with such a time lag between 2.4.x releases (previously), and (b) no 
2.6.0 kernel yet, it is understandable that driver authors are 
concentrating on 2.4, because that's where the users are right now.

What do to?

Currently we are really operating on the "hope and pray" system.  Nobody 
(IMO) really pays much attention to the API changes, since the 2.4.x 
releases were so far apart.  Between releases, "features depending on 
features depending features" situations were created.  However, since 
Marcelo has fixed this problem, I feel that some reflection on what a 
stable series means is called for.

Does it mean, no API changes except for security (or similarly severe) bugs?
Does it mean, no userland ABI changes, but API changes affecting onto 
the kernel are ok?
Does it mean, "just don't break things such that people are pissed off"?

I request the community's input, particularly those CC'd, for some sort 
of direction or consensus on this.

In any case, I personally feel that increased stability of the API, 
coupled with the increased frequency of releases, will make the most 
people happy.  I would prefer some sort of 2.4.x API freeze, say 
post-2.4.22, but maybe that's too radical?

	Jeff

Re: RFC: what's in a stable series?

[Jeff Garzik]

Jeff Garzik wrote:
> Does it mean, no userland ABI changes, but API changes affecting onto 
> the kernel are ok?

s/onto/only/  :)

Re: RFC: what's in a stable series?

[Herbert Pötzl]

On Wed, Jul 09, 2003 at 09:06:16PM -0400, Jeff Garzik wrote:
> So, I was wondering if it is possible to find a consensus 
> on what sorts of changes are ok in a stable series, 
> as it matures.  This mainly applies to 2.4 right now, 
> but I am interested in thoughts on 2.6.x also.
> 
> What do to?

In my opinion (and you requested input *g*), the
kernel userland API can be changed as much as is
required to improve/stabilize/bugfix the kernel,
unless this change breaks something in userland
without an already available update/upgrade/etc ...

To me it makes no sense to keep an API or some
kind of compatibilty thing alive, just because
it was defined somewhere along the way, and some
programmer might have picked it up ...

when (for example) I decide to use devfs at boot
time, completely replacing the traditional /dev
I will certainly break some scripts/applications, 
but it is my decision to do so. The same is true 
for any kernel update, except for security bugfixes, 
which are pretty well handled by the big distros
anyway ...

In general, I would prefer some -preX series, 
introducing a new feature or API, hanging around
some time (two to three weeks) with occasional
code cleanups, waiting for somebody shouting 
"hey guys, this stuff breaks my famous wossname".

In the end either the new feature is silently
accepted, or backed out, starting a new cycle of 
feature add/feature settle ...

> Does it mean, no API changes except for security 
> (or similarly severe) bugs?

security issues should be addressed in any case.

> Does it mean, no userland ABI changes, but API changes 
> affecting only the kernel are ok?

if some change affects one or two driver 
developer (read only a few people), then they
should agree on a change and let the people
test whatever they've agreed on ...

> Does it mean, "just don't break things such that people are pissed off"?

I guess this will be the most favored answer 
from any user ... and at least some developers ...

> I request the community's input, particularly those 
> CC'd, for some sort of direction or consensus on this.
> 
> In any case, I personally feel that increased stability of the API, 
> coupled with the increased frequency of releases, will make the most 
> people happy.  I would prefer some sort of 2.4.x API freeze, say 
> post-2.4.22, but maybe that's too radical?

I guess this depends on the actual 2.6 time frame ...

best,
Herbert

Re: RFC: what's in a stable series?

[Christoph Hellwig]

On Thu, Jul 10, 2003 at 05:34:09AM +0200, Herbert Pötzl wrote:
> In my opinion (and you requested input *g*), the
> kernel userland API can be changed as much as is
> required to improve/stabilize/bugfix the kernel,
> unless this change breaks something in userland
> without an already available update/upgrade/etc ...

Changing the kernel/userland API and ABI is totally out of question - 
we _really_ can't do that.  This is about inkernel APIs for modules.

Re: RFC: what's in a stable series?

[Alan Cox]

On Iau, 2003-07-10 at 04:34, Herbert Pötzl wrote:
> In my opinion (and you requested input *g*), the
> kernel userland API can be changed as much as is
> required to improve/stabilize/bugfix the kernel,
> unless this change breaks something in userland
> without an already available update/upgrade/etc ...

In a lot of cases (like O_DIRECT) its cleaner to simply break the API
in a way that will spew warnings if people miss changes than mess around
with extra methods that instead break drivers that forgot to use C99
initializers.

I plan to carry on breaking the kernel internal API when I have to
and its easy to fix up the few affected users. I broke all the audio
drivers between 2.4.21->22 but that was worth doing for example.

Re: RFC: what's in a stable series?

[Herbert Pötzl]

On Thu, Jul 10, 2003 at 12:22:35PM +0100, Alan Cox wrote:
> On Iau, 2003-07-10 at 04:34, Herbert Pötzl wrote:
> > In my opinion (and you requested input *g*), the
> > kernel userland API can be changed as much as is
> > required to improve/stabilize/bugfix the kernel,
> > unless this change breaks something in userland
> > without an already available update/upgrade/etc ...
> 
> In a lot of cases (like O_DIRECT) its cleaner to simply break the API
> in a way that will spew warnings if people miss changes than mess around
> with extra methods that instead break drivers that forgot to use C99
> initializers.

yeah, I obviously forgot to mention this case ;)
and I agree that sometimes this is the best way
to do it ... 

but this is a little bit different because
a) you do it intentionally
b) you know how to fix it

> I plan to carry on breaking the kernel internal API when I have to
> and its easy to fix up the few affected users. I broke all the audio
> drivers between 2.4.21->22 but that was worth doing for example.

keep breaking the kernel internal ... 8-)

best,
Herbert

Re: RFC: what's in a stable series?

[Greg KH]

On Wed, Jul 09, 2003 at 09:06:16PM -0400, Jeff Garzik wrote:
> 
> Does it mean, no API changes except for security (or similarly severe) bugs?

Do that, and we will stagnate.

> Does it mean, no userland ABI changes, but API changes affecting onto 
> the kernel are ok?

That sounds acceptable to me.

But maybe I'm just biased, as I'm looking to start backporting some of
the USB 2.5 changes to 2.4 to fix real bugs that are in 2.4.  This will
require changing kernel apis.  And in doing so, fixing up all of the
in-kernel usages of that api will happen.

In doing this, it just enforces the fact that it really matters if you
have a in-kernel driver or not.  If you want to keep a driver or any
other kernel code outside of the main tree, it is costly, both in time
and effort.

My .02

greg k-h

Re: RFC: what's in a stable series?

[Marcelo Tosatti]

On Wed, 9 Jul 2003, Jeff Garzik wrote:

> Does it mean, no API changes except for security (or similarly severe) bugs?
> Does it mean, no userland ABI changes, but API changes affecting onto
> the kernel are ok?
> Does it mean, "just don't break things such that people are pissed off"?

My initial though on some new feature is to avoid it. We are a stable
release, after all.

The quota patches have been around for a long time, and Jan Kara has been
trying to include for sometime now (since 2.4.20/21). I tried to avoid it.

Now I realized the possible drawbacks of it are minimal (if any) compared
to the overall advantage it brings to Linux 2.4.

The same goes for ACPI, people have been trying to include it for a long
time, and I prefered to reject it until 2.4.22-pre.

Its a case-by-case problem.  You can't have a general rules like "no API
changes except for security (or similarly severe) bugs? Does it mean, no
userland ABI changes, but API changes affecting onto the kernel are ok?"

I reverted the direct IO patches because hch complained on me that they
change the direct IO API, and we really dont want that kind of
change, IMHO.

Trond/Christoph/you now have the direct_io2 possibility, which wouldnt
break the current API and still get us the desired "feature".

> I request the community's input, particularly those CC'd, for some sort
> of direction or consensus on this.
>
> In any case, I personally feel that increased stability of the API,
> coupled with the increased frequency of releases, will make the most
> people happy.  I would prefer some sort of 2.4.x API freeze, say
> post-2.4.22, but maybe that's too radical?

I also plan a 2.4.x API freeze. Maybe latter.

In general, I think each case is a case and has its tradeoffs which need
to be discussed and agreed on.

Re: RFC: what's in a stable series?

[Andrew Morton]

Marcelo Tosatti <marcelo@conectiva.com.br> wrote:
>
> Its a case-by-case problem.

It is.  Generally I think we should prefer to do the right thing rather
than adhering to the old API out of some principle.

Evaluate the impact on out-of-tree kernel patches (especially vendor
kernels) and if it is unacceptable then reject the change or augment the API
rather than changing it.

>  I reverted the direct IO patches because hch complained on me that they
>  change the direct IO API, and we really dont want that kind of
>  change, IMHO.

OK, we're on to a specific case.  Albeit a very small one.

I think Trond's direct IO change was right.  The impact on out-of-tree code
is infinitesimal.  Stick a #define O_DIRECT_NEEDS_A_FILP in the header and
let the XFS guys write a four-line patch.  There's no point in mucking up
the kernel API to save such a small amount of work.

Or merge XFS.

Re: RFC: what's in a stable series?

[Christoph Hellwig]

On Wed, Jul 09, 2003 at 09:16:45PM -0700, Andrew Morton wrote:
> >  I reverted the direct IO patches because hch complained on me that they
> >  change the direct IO API, and we really dont want that kind of
> >  change, IMHO.
> 
> OK, we're on to a specific case.  Albeit a very small one.
> 
> I think Trond's direct IO change was right.  The impact on out-of-tree code
> is infinitesimal.  Stick a #define O_DIRECT_NEEDS_A_FILP in the header and
> let the XFS guys write a four-line patch.

Oh, we have that patch even without the feature define in say -ac and -aa
but it's just horrible to have APIs silently change behind you.  Especially
when just changing a function arg where you only get one more warning in
the forrest of warnings produced by gcc 3.3 on a 2.4 tree..

> Or merge XFS.

That's of course a good idea [1] but doesn't really help in this discussion.
There's other filesystems like ocfs or opengfs that have the same kind
of problems.

[1] and with the new quota code and vmap() we're almost there..

Re: RFC: what's in a stable series?

[Andrew Morton]

Christoph Hellwig <hch@infradead.org> wrote:
>
> Especially
>  when just changing a function arg where you only get one more warning in
>  the forrest of warnings produced by gcc 3.3 on a 2.4 tree..

Switching topics (to one of my favourites):

This is a perfect demonstration of why it is utterly unacceptable for
compiler developers to add new warnings without also providing a way of
turning them off.

Re: RFC: what's in a stable series?

[Christoph Hellwig]

On Thu, Jul 10, 2003 at 12:54:52AM -0300, Marcelo Tosatti wrote:
> The quota patches have been around for a long time, and Jan Kara has been
> trying to include for sometime now (since 2.4.20/21). I tried to avoid it.
> 
> Now I realized the possible drawbacks of it are minimal (if any) compared
> to the overall advantage it brings to Linux 2.4.

Also the quota patches don't change any ABI or API - userland can still
use the old ABI in addition to the new one, 16bit ondisk quotas are
still supported and filesystems couldn't care less which implementation
it plugs into - the API is the same.

Re: RFC: what's in a stable series?

[Alan Cox]

On Iau, 2003-07-10 at 08:53, Christoph Hellwig wrote:
> Also the quota patches don't change any ABI or API - userland can still
> use the old ABI in addition to the new one, 16bit ondisk quotas are
> still supported and filesystems couldn't care less which implementation
> it plugs into - the API is the same.

Because you hacked v1 support out of Jan Kara's stuff the quota bits are
pretty useless to most people because they have v1 format files.

Re: RFC: what's in a stable series?

[Marcelo Tosatti]

Linus removed from CC because I think he does not care about 2.4.x
specific issues - which is what this thread is now about.

On Thu, 10 Jul 2003, Alan Cox wrote:

> On Iau, 2003-07-10 at 08:53, Christoph Hellwig wrote:
> > Also the quota patches don't change any ABI or API - userland can still
> > use the old ABI in addition to the new one, 16bit ondisk quotas are
> > still supported and filesystems couldn't care less which implementation
> > it plugs into - the API is the same.
>
> Because you hacked v1 support out of Jan Kara's stuff the quota bits are
> pretty useless to most people because they have v1 format files.

So Christoph's quota patch does not support vendors "v1" files?

I must be misunderstanding someone.

Re: RFC: what's in a stable series?

[Alan Cox]

On Iau, 2003-07-10 at 13:13, Marcelo Tosatti wrote:
> So Christoph's quota patch does not support vendors "v1" files?
> 
> I must be misunderstanding someone.

There are three species of quota in Linux

v0	(official old Linux)
v1	(most 2.4 vendor trees)
v2	(the 2.5 format)

[Plus other per file system stuff that is abstracted by the quota
updates and used by folks like XFS]

The v1 quota definitely doesn't belong in 2.5, but its needed by a
lot of 2.4 users who come from vendor trees (and may want to go back
again)

Re: RFC: what's in a stable series?

[Christoph Hellwig]

On Thu, Jul 10, 2003 at 01:42:00PM +0100, Alan Cox wrote:
> On Iau, 2003-07-10 at 13:13, Marcelo Tosatti wrote:
> > So Christoph's quota patch does not support vendors "v1" files?
> > 
> > I must be misunderstanding someone.
> 
> There are three species of quota in Linux
> 
> v0	(official old Linux)
> v1	(most 2.4 vendor trees)
> v2	(the 2.5 format)

Umm, no.  You misunderstood.

There's two quota _ondisk_ formats:

v1 old 16bit quota.
v2 new 32bit quota.

2.4 previously only supported v1, 2.5 and 2.4.22-pre4 also support v2.

There's three sys_quotactl ABIs

1) original 16 bit one, supported by all kernels <= 2.4
2) first 32bit one, supported by 2.4-ac any many vendor trees, but never
   in mainline
3) new 32bit one, supported by 2.4.x (x >= 22-pre4) and 2.5

Unfortunately the second one uses the same constants as the old 16bit one
but different structures so there is no way to support both in a single
kernel.

Re: RFC: what's in a stable series?

[Jan Kara]

> On Thu, Jul 10, 2003 at 01:42:00PM +0100, Alan Cox wrote:
> > On Iau, 2003-07-10 at 13:13, Marcelo Tosatti wrote:
> > > So Christoph's quota patch does not support vendors "v1" files?
> > > 
> > > I must be misunderstanding someone.
> > 
> > There are three species of quota in Linux
> > 
> > v0	(official old Linux)
> > v1	(most 2.4 vendor trees)
> > v2	(the 2.5 format)
> 
> Umm, no.  You misunderstood.
> 
> There's two quota _ondisk_ formats:
> 
> v1 old 16bit quota.
> v2 new 32bit quota.
> 
> 2.4 previously only supported v1, 2.5 and 2.4.22-pre4 also support v2.
  Exactly.

> There's three sys_quotactl ABIs
> 
> 1) original 16 bit one, supported by all kernels <= 2.4
> 2) first 32bit one, supported by 2.4-ac any many vendor trees, but never
>    in mainline
  Actually at least later SuSE kernels and -ac kernels support 3)...

> 3) new 32bit one, supported by 2.4.x (x >= 22-pre4) and 2.5
  Right.

> Unfortunately the second one uses the same constants as the old 16bit one
> but different structures so there is no way to support both in a single
> kernel.

								Honza

-- 
Jan Kara <jack@suse.cz>
SuSE CR Labs

Re: RFC: what's in a stable series?

[Christoph Hellwig]

On Thu, Jul 10, 2003 at 09:00:30PM +0200, Jan Kara wrote:
> > 1) original 16 bit one, supported by all kernels <= 2.4
> > 2) first 32bit one, supported by 2.4-ac any many vendor trees, but never
> >    in mainline
>   Actually at least later SuSE kernels and -ac kernels support 3)...

And the latest 2.4.20-based Red Hat kernel (RH 7/8/9 errata) support
_only_ 3, and the 32bit ondisk quota format.

Re: RFC: what's in a stable series?

[Alan Cox]

On Iau, 2003-07-10 at 16:15, Christoph Hellwig wrote:
> Unfortunately the second one uses the same constants as the old 16bit one
> but different structures so there is no way to support both in a single
> kernel.

Ok that makes a lot more sense

Re: RFC: what's in a stable series?

[Christoph Hellwig]

On Thu, Jul 10, 2003 at 09:13:03AM -0300, Marcelo Tosatti wrote:
> > Because you hacked v1 support out of Jan Kara's stuff the quota bits are
> > pretty useless to most people because they have v1 format files.
> 
> So Christoph's quota patch does not support vendors "v1" files?
> 
> I must be misunderstanding someone.

Alan obviously didn't even look at the code.  Of course the old
quota format is supported as described in the mail I sent you
the patch with.

Re: RFC: what's in a stable series?

[Christoph Hellwig]

On Thu, Jul 10, 2003 at 12:19:59PM +0100, Alan Cox wrote:
> Because you hacked v1 support out of Jan Kara's stuff the quota bits are
> pretty useless to most people because they have v1 format files.

Please take a look at the code before saying something that unqalified. (
or just read the changelog..).

The old quota format is enabled unconditionally, not removed.