Development tree, PLEASE?

Development tree, PLEASE?

[Michael Loftis]

OK, I don't know abotu others, but I'm starting to get sick of this 
unstable stable kernel.  Either change the statements allover that were 
made that even-numbered kernels were going to be stable or open 2.7. 
Removing devfs has profound effects on userland.  It's one thing to screw 
with all of the embedded developers, nearly all kernel module developers, 
etc, by changing internal APIs but this is completely out of hand.

Normally I wouldn't care, and I'd just stay away from 'stable' until 
someone finally figured out that a dev tree really is needed, but I can't 
stay quiet anymore.  2.6.x is anything but stable right now.  It might be 
stable in the sense that most any development kernel is stable in that it 
runs without crashing, but it's not at all stable in the sense that 
everything is changing as if it were an odd numbered dev tree.

Yes, I'm venting some frustrations here, but I can't be the only one.  I 
know now I'm going to be called a troll or a naysayer but whatever.  The 
fact is it needs saying.  I shouldn't have to do major changes to 
accomodate sysfs in a *STABLE* kernel when going between point revs.  This 
is just not how it's been done in the past.

I can sympathize with the need to push code out to users faster, and to 
simplify maintenance as LK is a huge project, but at the expense of how 
many developers?



--
"Genius might be described as a supreme capacity for getting its possessors
into trouble of all kinds."
-- Samuel Butler

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 20, 2006 8:17:40 AM -0700 Michael Loftis <mloftis@wgops.com> 
wrote:

> I can sympathize with the need to push code out to users faster, and to
> simplify maintenance as LK is a huge project, but at the expense of how
> many developers?


I'll just quickly clarify that last statement, I meant userland developers 
and distro developers.

Re: Development tree, PLEASE?

[Marc Koschewski]

* Michael Loftis <mloftis@wgops.com> [2006-01-20 08:17:40 -0700]:

> OK, I don't know abotu others, but I'm starting to get sick of this 
> unstable stable kernel.  Either change the statements allover that were 
> made that even-numbered kernels were going to be stable or open 2.7. 
> Removing devfs has profound effects on userland.  It's one thing to screw 
> with all of the embedded developers, nearly all kernel module developers, 
> etc, by changing internal APIs but this is completely out of hand.
> 
> Normally I wouldn't care, and I'd just stay away from 'stable' until 
> someone finally figured out that a dev tree really is needed, but I can't 
> stay quiet anymore.  2.6.x is anything but stable right now.  It might be 
> stable in the sense that most any development kernel is stable in that it 
> runs without crashing, but it's not at all stable in the sense that 
> everything is changing as if it were an odd numbered dev tree.
> 
> Yes, I'm venting some frustrations here, but I can't be the only one.  I 
> know now I'm going to be called a troll or a naysayer but whatever.  The 
> fact is it needs saying.  I shouldn't have to do major changes to 
> accomodate sysfs in a *STABLE* kernel when going between point revs.  This 
> is just not how it's been done in the past.
> 
> I can sympathize with the need to push code out to users faster, and to 
> simplify maintenance as LK is a huge project, but at the expense of how 
> many developers?

For my daily work I use the -git kernels on my production machines. And I didn't
have probs for a long time due to stuff being tested in -mm. -mm is mostly
broken for me (psmouse, now reiserfs, ...) but I tend to say that 2.6 is
rock-stable.

When it comes to API stability people are encouraged to stay up-to-date when
when developing stuff out of the kernel tree, ain't they? A more convenient way
would be to create a new in-kernel-tree wrapper module that wraps some functions
no longer available anymore and which are possible to be wrapped in the meaning
of all needed data is provided to the 'old' method and can be easyily wrapped
into the new function.

It could a Kconfig option so that the 'wrapper module' is only activated on
demand. Thus, having the option to port driver directly to the new API or just
silenty use the 'wrapper module' to translate the call while being noisy at
compile time.

You're free to write the module... ;)

Marc

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 20, 2006 4:59:19 PM +0100 Marc Koschewski 
<marc@osknowledge.org> wrote:

> For my daily work I use the -git kernels on my production machines. And I
> didn't have probs for a long time due to stuff being tested in -mm. -mm
> is mostly broken for me (psmouse, now reiserfs, ...) but I tend to say
> that 2.6 is rock-stable.
>
> When it comes to API stability people are encouraged to stay up-to-date
> when when developing stuff out of the kernel tree, ain't they? A more
> convenient way would be to create a new in-kernel-tree wrapper module
> that wraps some functions no longer available anymore and which are
> possible to be wrapped in the meaning of all needed data is provided to
> the 'old' method and can be easyily wrapped into the new function.
>
> It could a Kconfig option so that the 'wrapper module' is only activated
> on demand. Thus, having the option to port driver directly to the new API
> or just silenty use the 'wrapper module' to translate the call while
> being noisy at compile time.
>
> You're free to write the module... ;)

I know this, but the in-kernel stuff is far less painful than when all this 
stuff bleeds to userland.  Besides, can you imagine such a beast of a 
module? :D  I mean yeouch, the maintenance.  And it'd encourage the sort of 
thing we as kernel developers in general want to discourage, which is the 
use of deprecated APIs.

Lots of things still out there depend on devfs.  So now if I want to 
develop my kmod on recent kernels I have to be in the business of 
maintaining a lot more userland stuff, like mkinitrd, installers, etc. that 
have come to rely on devfs.

The point is this is happening in what is being called a stable kernel. 
Stable it isn't.  The whole devfs thing is likely to cause me a lot of 
work, I'd stay with 2.4 in the worst affected things, but I can't.  devfs 
is newish for and already been deprecated and killed before a major release 
even, that just seems not right.

Anyway hopefully this thread wills tart some constructive thought on this 
rather than being pointless, but I had to get it out there.  I know I have 
a habit of showing up every other year or so. :)

Re: Development tree, PLEASE?

[James Courtier-Dutton]

Michael Loftis wrote:

> OK, I don't know abotu others, but I'm starting to get sick of this 
> unstable stable kernel.  Either change the statements allover that 
> were made that even-numbered kernels were going to be stable or open 
> 2.7. Removing devfs has profound effects on userland.  It's one thing 
> to screw with all of the embedded developers, nearly all kernel module 
> developers, etc, by changing internal APIs but this is completely out 
> of hand.
>
> Normally I wouldn't care, and I'd just stay away from 'stable' until 
> someone finally figured out that a dev tree really is needed, but I 
> can't stay quiet anymore.  2.6.x is anything but stable right now.  It 
> might be stable in the sense that most any development kernel is 
> stable in that it runs without crashing, but it's not at all stable in 
> the sense that everything is changing as if it were an odd numbered 
> dev tree.
>
> Yes, I'm venting some frustrations here, but I can't be the only one.  
> I know now I'm going to be called a troll or a naysayer but whatever.  
> The fact is it needs saying.  I shouldn't have to do major changes to 
> accomodate sysfs in a *STABLE* kernel when going between point revs.  
> This is just not how it's been done in the past.
>
> I can sympathize with the need to push code out to users faster, and 
> to simplify maintenance as LK is a huge project, but at the expense of 
> how many developers?


It is unclear what you are really ranting about here. The "stable" 
kernel is stable or at least as stable as it is going to be. It is left 
to distros to make it even more stable. The interface to user land has 
not changed.
If all you are ranting about is the move from devfs to udevd, then all 
the user land tools dealing with them have been updated already.

What is the real specific problem you are having?

James

Re: Development tree, PLEASE?

[Marc Koschewski]

* Michael Loftis <mloftis@wgops.com> [2006-01-20 09:07:16 -0700]:

> 
> 
> --On January 20, 2006 4:59:19 PM +0100 Marc Koschewski 
> <marc@osknowledge.org> wrote:
> 
> >For my daily work I use the -git kernels on my production machines. And I
> >didn't have probs for a long time due to stuff being tested in -mm. -mm
> >is mostly broken for me (psmouse, now reiserfs, ...) but I tend to say
> >that 2.6 is rock-stable.
> >
> >When it comes to API stability people are encouraged to stay up-to-date
> >when when developing stuff out of the kernel tree, ain't they? A more
> >convenient way would be to create a new in-kernel-tree wrapper module
> >that wraps some functions no longer available anymore and which are
> >possible to be wrapped in the meaning of all needed data is provided to
> >the 'old' method and can be easyily wrapped into the new function.
> >
> >It could a Kconfig option so that the 'wrapper module' is only activated
> >on demand. Thus, having the option to port driver directly to the new API
> >or just silenty use the 'wrapper module' to translate the call while
> >being noisy at compile time.
> >
> >You're free to write the module... ;)
> 
> I know this, but the in-kernel stuff is far less painful than when all this 
> stuff bleeds to userland.  Besides, can you imagine such a beast of a 
> module? :D  I mean yeouch, the maintenance.  And it'd encourage the sort of 
> thing we as kernel developers in general want to discourage, which is the 
> use of deprecated APIs.

I know. But you we're the one who started this off. The 'beast' should be
maintained by people that need it. And it was just a brainstorm, moreover.

> 
> Lots of things still out there depend on devfs.  So now if I want to 
> develop my kmod on recent kernels I have to be in the business of 
> maintaining a lot more userland stuff, like mkinitrd, installers, etc. that 
> have come to rely on devfs.

What exactly relies on _devfs_?

> 
> The point is this is happening in what is being called a stable kernel. 
> Stable it isn't.  The whole devfs thing is likely to cause me a lot of 
> work, I'd stay with 2.4 in the worst affected things, but I can't.  devfs 
> is newish for and already been deprecated and killed before a major release 
> even, that just seems not right.

As far as I remember the devfs maintainer didn't do just a one-liner of changes
plus he was not to be reached by mail. No reply, no list acitivity, ... nothing.
Just out of the town.

> 
> Anyway hopefully this thread wills tart some constructive thought on this 
> rather than being pointless, but I had to get it out there.  I know I have 
> a habit of showing up every other year or so. :)

This thread will start something, yes. But I don't think we will have a decision
in the end. The kernel grows. In size, in features, in just about anything. And
from a developers point of view it's always wise to re-write a subsystem with a
new API and the freedom to do _whatever_ one thinks she could do than re-write a
subsystem due to maintaining the interface for compatibility.

The two cases exactly are:

	* _new_ code with all new features planned

	or

	* _partly new_ code with some new features due to API incompatibility a la
	  'what we have to do is to get the best we can from what we have'

And the latter is definitely the wrong way to go. Just my 2 cents...

Marc

Re: Development tree, PLEASE?

[Marc Koschewski]

* Michael Loftis <mloftis@wgops.com> [2006-01-20 09:07:16 -0700]:

> 
> 
> --On January 20, 2006 4:59:19 PM +0100 Marc Koschewski 
> <marc@osknowledge.org> wrote:
> 
> >For my daily work I use the -git kernels on my production machines. And I
> >didn't have probs for a long time due to stuff being tested in -mm. -mm
> >is mostly broken for me (psmouse, now reiserfs, ...) but I tend to say
> >that 2.6 is rock-stable.
> >
> >When it comes to API stability people are encouraged to stay up-to-date
> >when when developing stuff out of the kernel tree, ain't they? A more
> >convenient way would be to create a new in-kernel-tree wrapper module
> >that wraps some functions no longer available anymore and which are
> >possible to be wrapped in the meaning of all needed data is provided to
> >the 'old' method and can be easyily wrapped into the new function.
> >
> >It could a Kconfig option so that the 'wrapper module' is only activated
> >on demand. Thus, having the option to port driver directly to the new API
> >or just silenty use the 'wrapper module' to translate the call while
> >being noisy at compile time.
> >
> >You're free to write the module... ;)
> 
> I know this, but the in-kernel stuff is far less painful than when all this 
> stuff bleeds to userland.  Besides, can you imagine such a beast of a 
> module? :D  I mean yeouch, the maintenance.  And it'd encourage the sort of 
> thing we as kernel developers in general want to discourage, which is the 
> use of deprecated APIs.
> 
> Lots of things still out there depend on devfs.  So now if I want to 
> develop my kmod on recent kernels I have to be in the business of 
> maintaining a lot more userland stuff, like mkinitrd, installers, etc. that 
> have come to rely on devfs.

Moreover, as far as I remember... my devfsd -> udev transsition went as smooth
as a reboot.

Marc

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 20, 2006 4:29:44 PM +0000 James Courtier-Dutton 
<James@superbug.co.uk> wrote:

> It is unclear what you are really ranting about here. The "stable" kernel
> is stable or at least as stable as it is going to be. It is left to
> distros to make it even more stable. The interface to user land has not
> changed.
> If all you are ranting about is the move from devfs to udevd, then all
> the user land tools dealing with them have been updated already.

That's the nail on the head exactly.  Why is this being done in an even 
numbered kernel?  This represents an API change that has knock on well 
outside of the kernel, and should be done in development releases.  Why is 
it LK is the only major project (that I know of) that does this?  This is 
akin to apache changing the format of httpd.conf and saying in say 1.3.38 
and saying 'well we made the userland tools too.'

>
> What is the real specific problem you are having?

Well there's a whole grab bag of them that I'll be getting to over the next 
few months, but the most immediate is the fact that I've gotten new 
hardware from a venduh that requires me to build a new Debian installer and 
new debian kernels.  I also have custom packages that depend on devfs being 
there and now it's not.

Yes I realise this change isn't out of the blue or anything, but it's in a 
'stable' kernel.  Why bother calling 2.6 stable?  We may as well have 
stayed at 2.5 if this sort of thing is going to continue to be pulled.

>
> James
>
>



--
"Genius might be described as a supreme capacity for getting its possessors
into trouble of all kinds."
-- Samuel Butler

Re: Development tree, PLEASE?

[Dmitry Torokhov]

On 1/20/06, Marc Koschewski <marc@osknowledge.org> wrote:
>
> For my daily work I use the -git kernels on my production machines. And I didn't
> have probs for a long time due to stuff being tested in -mm. -mm is mostly
> broken for me (psmouse, now reiserfs, ...) but I tend to say that 2.6 is
> rock-stable.
>

[Trying to steal the thread somewhat...]

Marc, have you tried 2.6.16-rc1 yet? Does it also give you problems
with psmouse?

--
Dmitry

Re: Development tree, PLEASE?

[Jan Engelhardt]

> Lots of things still out there depend on devfs.  So now if I want to develop my
> kmod on recent kernels I have to be in the business of maintaining a lot more
> userland stuff, like mkinitrd, installers, etc. that have come to rely on
> devfs.

Just like the OSS-ALSA/e100 debate: If there IS something that you 
do not like [something that requires devfs], why has NO ONE objected? 
(Quoting Greg: "and I have not heard a single peep out of anyone about the 
email titled "Subject: devfs going away, last chance to complain"") Not to 
forget there is ndevfs if you really need it.


Jan Engelhardt
-- 

Re: Development tree, PLEASE?

[Marc Koschewski]

* Dmitry Torokhov <dmitry.torokhov@gmail.com> [2006-01-20 11:40:42 -0500]:

> On 1/20/06, Marc Koschewski <marc@osknowledge.org> wrote:
> >
> > For my daily work I use the -git kernels on my production machines. And I didn't
> > have probs for a long time due to stuff being tested in -mm. -mm is mostly
> > broken for me (psmouse, now reiserfs, ...) but I tend to say that 2.6 is
> > rock-stable.
> >
> 
> [Trying to steal the thread somewhat...]
> 
> Marc, have you tried 2.6.16-rc1 yet? Does it also give you problems
> with psmouse?
> 

Not yet, Dmitry. I just managed to get today's -git compiled and running. I'll
give it a try tonite.

Moreover, I put a

	echo -n 0 > /sys/bus/serio/devices/serio0/resync_time

in my initscripts. Since then I didn't see any problem. I'll report later how it
went with that line removed and stock 2.6.16-rc1.

Marc

Re: Development tree, PLEASE?

[Dmitry Torokhov]

On 1/20/06, Michael Loftis <mloftis@wgops.com> wrote:
>
> Well there's a whole grab bag of them that I'll be getting to over the next
> few months, but the most immediate is the fact that I've gotten new
> hardware from a venduh that requires me to build a new Debian installer and
> new debian kernels.  I also have custom packages that depend on devfs being
> there and now it's not.
>
> Yes I realise this change isn't out of the blue or anything, but it's in a
> 'stable' kernel.  Why bother calling 2.6 stable?  We may as well have
> stayed at 2.5 if this sort of thing is going to continue to be pulled.
>

Ok, so you agree that there was an ample warning that devfs is going
away... Now, what would be different if 2.8.0 released tomorrow
without devfs and your vendor would require you to build new Debian
installer and kernel?

--
Dmitry

Re: Development tree, PLEASE?

[Joe George]

Michael Loftis wrote:
> 
> 
> --On January 20, 2006 4:29:44 PM +0000 James Courtier-Dutton 
> <James@superbug.co.uk> wrote:
> 
>> It is unclear what you are really ranting about here. The "stable" kernel
>> is stable or at least as stable as it is going to be. It is left to
>> distros to make it even more stable. The interface to user land has not
>> changed.
>> If all you are ranting about is the move from devfs to udevd, then all
>> the user land tools dealing with them have been updated already.
> 
> That's the nail on the head exactly.  Why is this being done in an even 
> numbered kernel?  This represents an API change that has knock on well 
> outside of the kernel, and should be done in development releases.  Why 
> is it LK is the only major project (that I know of) that does this?  
> This is akin to apache changing the format of httpd.conf and saying in 
> say 1.3.38 and saying 'well we made the userland tools too.'
> 
>>
>> What is the real specific problem you are having?
> 
> Well there's a whole grab bag of them that I'll be getting to over the 
> next few months, but the most immediate is the fact that I've gotten new 
> hardware from a venduh that requires me to build a new Debian installer 
> and new debian kernels.  I also have custom packages that depend on 
> devfs being there and now it's not.
> 
> Yes I realise this change isn't out of the blue or anything, but it's in 
> a 'stable' kernel.  Why bother calling 2.6 stable?  We may as well have 
> stayed at 2.5 if this sort of thing is going to continue to be pulled.
> 

I don't think that kernel developers are calling 2.6 a stable kernel
series.  There was an evolution into another development model without
a corresponding change in the kernel numbering.  I think the main
reason the numbering wasn't changed was that it would break thousands
of scripts people are using all over the world.

What would be nice is to go, for example, from 2.6.17 to 3.1, 3.2,
3.3, ...  And have what is currently called the stable series start at
3.1.1.  This would make it clear that the 2.4/2.5 way of doing business
is over.  Someone would have to decide whether it is worth it to break
all the scripts, however.

Joe

Re: Development tree, PLEASE?

[Dmitry Torokhov]

On 1/20/06, Marc Koschewski <marc@osknowledge.org> wrote:
> * Dmitry Torokhov <dmitry.torokhov@gmail.com> [2006-01-20 > >
> > Marc, have you tried 2.6.16-rc1 yet? Does it also give you problems
> > with psmouse?
> >
>
> Not yet, Dmitry. I just managed to get today's -git compiled and running. I'll
> give it a try tonite.
>
> Moreover, I put a
>
>        echo -n 0 > /sys/bus/serio/devices/serio0/resync_time
>
> in my initscripts. Since then I didn't see any problem. I'll report later how it
> went with that line removed and stock 2.6.16-rc1.
>

Ok, if psmouse gives you a fit and setting resync_time to 0 cures it,
please do the follwing:

echo -n 5 > /sys/bus/serio/devices/serioX/resync_time
echo 1 > /sys/modules/i8042/parameters/debug
... wait 10 seconds ...
move the mouse slightly
... wait another 10 seconds ...
move the mouse slighty again
echo 0 > /sys/modules/i8042/parameters/debug

and send me your dmesg (or better /var/log/messages or whatever file
you use for kernel messages).

--
Dmitry

Re: Development tree, PLEASE?

[Randy.Dunlap]

On Fri, 20 Jan 2006, Joe George wrote:

> Michael Loftis wrote:
> >
> >
> > --On January 20, 2006 4:29:44 PM +0000 James Courtier-Dutton
> > <James@superbug.co.uk> wrote:
> >
> >> It is unclear what you are really ranting about here. The "stable" kernel
> >> is stable or at least as stable as it is going to be. It is left to
> >> distros to make it even more stable. The interface to user land has not
> >> changed.
> >> If all you are ranting about is the move from devfs to udevd, then all
> >> the user land tools dealing with them have been updated already.
> >
> > That's the nail on the head exactly.  Why is this being done in an even
> > numbered kernel?  This represents an API change that has knock on well
> > outside of the kernel, and should be done in development releases.  Why
> > is it LK is the only major project (that I know of) that does this?
> > This is akin to apache changing the format of httpd.conf and saying in
> > say 1.3.38 and saying 'well we made the userland tools too.'
> >
> >>
> >> What is the real specific problem you are having?
> >
> > Well there's a whole grab bag of them that I'll be getting to over the
> > next few months, but the most immediate is the fact that I've gotten new
> > hardware from a venduh that requires me to build a new Debian installer
> > and new debian kernels.  I also have custom packages that depend on
> > devfs being there and now it's not.
> >
> > Yes I realise this change isn't out of the blue or anything, but it's in
> > a 'stable' kernel.  Why bother calling 2.6 stable?  We may as well have
> > stayed at 2.5 if this sort of thing is going to continue to be pulled.
> >
>
> I don't think that kernel developers are calling 2.6 a stable kernel
> series.  There was an evolution into another development model without
> a corresponding change in the kernel numbering.  I think the main
> reason the numbering wasn't changed was that it would break thousands
> of scripts people are using all over the world.
>
> What would be nice is to go, for example, from 2.6.17 to 3.1, 3.2,
> 3.3, ...  And have what is currently called the stable series start at
> 3.1.1.  This would make it clear that the 2.4/2.5 way of doing business
> is over.  Someone would have to decide whether it is worth it to break
> all the scripts, however.

The problems AFAICT are:

1.  We did (for 2.5/2.4) or would (for 3.3/3.2) spend tons of time
in backporting new features or drivers from the development tree
to the stable tree.  The current model saves that duplication
(or even worse if multiple distros do that same work).

2.  If we did have a separate development tree, we would need
to clone Andrew. 8:)  IMO there aren't a lot of choices for qualified
tree maintainers, although I'm sure we could find someone if we
had to.

Anyway, to summarize, it's about manpower and efficient use of it.

-- 
~Randy

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 20, 2006 5:34:33 PM +0100 Marc Koschewski 
<marc@osknowledge.org> wrote:

> I know. But you we're the one who started this off. The 'beast' should be
> maintained by people that need it. And it was just a brainstorm, moreover.

Understood.  However these sorts of changes are still more appropriate in a 
development kernel/tree, not in what's generally supposed to be accepted as 
a stable code base.

>> Lots of things still out there depend on devfs.  So now if I want to
>> develop my kmod on recent kernels I have to be in the business of
>> maintaining a lot more userland stuff, like mkinitrd, installers, etc.
>> that  have come to rely on devfs.
>
> What exactly relies on _devfs_?

devfs=mount/nomount for one in kernel params, mount commands for another 
(mount -t devfs ....), modprobe devfs, these are at the lowest level. 
Scripts are written, as are bits of code, with the assumption that they'll 
get their /dev tree/dependency satisfied in a certain way.

>
>>
>> The point is this is happening in what is being called a stable kernel.
>> Stable it isn't.  The whole devfs thing is likely to cause me a lot of
>> work, I'd stay with 2.4 in the worst affected things, but I can't.
>> devfs  is newish for and already been deprecated and killed before a
>> major release  even, that just seems not right.
>
> As far as I remember the devfs maintainer didn't do just a one-liner of
> changes plus he was not to be reached by mail. No reply, no list
> acitivity, ... nothing. Just out of the town.
>
>>
>> Anyway hopefully this thread wills tart some constructive thought on
>> this  rather than being pointless, but I had to get it out there.  I
>> know I have  a habit of showing up every other year or so. :)
>
> This thread will start something, yes. But I don't think we will have a
> decision in the end. The kernel grows. In size, in features, in just
> about anything. And from a developers point of view it's always wise to
> re-write a subsystem with a new API and the freedom to do _whatever_ one
> thinks she could do than re-write a subsystem due to maintaining the
> interface for compatibility.

Which is fine in a development tree.  The problem here is making these 
changes, blowing away APIs, especially userland ones, in a stable tree. 
For various reasons embedded systems will often need to stay current with 
the 'stable' kernel.  You try developing modules against a 'stable' kernel 
for embedded purposes when things are changing under you.  Yes that's a 
partially commercial argument, but it's also a general argument.  The even 
numbered releases are/were supposed to be atleast mostly stable.  And in my 
mind atleast that means that APIs don't come and go on a whim.

> The two cases exactly are:
>
> 	* _new_ code with all new features planned
>
> 	or
>
> 	* _partly new_ code with some new features due to API incompatibility a
> la 	  'what we have to do is to get the best we can from what we have'
>
> And the latter is definitely the wrong way to go. Just my 2 cents...

Which is why everyone else seperates development from maintenance.  AIX, 
HP-UX, even Windows does (ok...so maybe not Windows).

The problem is these sorts of changes are 'normally' reserved for 
development trees because they can break (and will) break things and they 
obviously change things.

What do you think would happen if OpenSSL changed it's API incompatibly in 
say (arbitrarily) six months for a point release doing away with something 
in a stable release?

I guess I'm just the net.kook and the only one spending time and effort to 
clean up a mess created simply because he needed to go up a point rev, and 
atleast one otherwise working feature.

No devfs was not, is not perfect, and I agree something better was needed, 
but in a stable kernel?  I'm arguing against userland API changes 
specifically in stable kernels, and in a more broad sense even internal API 
changes in a stable kernel, atleast where they most certainly affect 
development of code, or maintenance releases of code requiring upgrades of 
the kernel in general.

If 2.6 were stable I should be able to write a module for the '2.6' API, 
like PHP, like Apache, like Zend, like OpenSSL, and be reasonably certain 
that atleast the API wasn't going to drastically change or go away when I 
rebuild the binaries for a security update or 'minor' update of say needing 
support for the new PCI IDs of the latest e1000 variants.

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 20, 2006 5:35:51 PM +0100 Marc Koschewski 
<marc@osknowledge.org> wrote:

> Moreover, as far as I remember... my devfsd -> udev transsition went as
> smooth as a reboot.

The one machine I've got running 2.6+devfs under debian chokes in initrd 
with an inability to find devfs during boot so I had to go back to static 
/dev entries for it since atleast in sarge right now I'm not seeing a 
quick-and-easy way to get devfs like support bundled via mkinitrd, but I 
haven't looked, and I shouldn't have to.  It shouldn't have gone away in a 
stable kernel in teh first place.  I realise things are changing, and need 
to change but that's called development, and belongs in a development tree.

>
> Marc
>



--
"Genius might be described as a supreme capacity for getting its possessors
into trouble of all kinds."
-- Samuel Butler

Re: Development tree, PLEASE?

[Gábor Lénárt]

Though I'm not a kernel developer let me allow to comment this based on
my experiences as well.

On Fri, Jan 20, 2006 at 08:17:40AM -0700, Michael Loftis wrote:
> OK, I don't know abotu others, but I'm starting to get sick of this 
> unstable stable kernel.  Either change the statements allover that were 

What kind of instability have you got? I haven't had any instability since
at least a year or so, or if there was it was some kind of hardware fault.
In fact, many machines (like an Armada E500 notebook and some servers as well)
seems to be stable which was NOT in case of 2.4 kernels! So for our experience
at our workplace, 2.6 seems to be much more usable than 2.4.x kernels (ok,
it may be caused by "newer" hardwares, on quite old machines I can't show
major difference in stability between 2.4 and 2.6)

> made that even-numbered kernels were going to be stable or open 2.7. 
> Removing devfs has profound effects on userland.  It's one thing to screw 
> with all of the embedded developers, nearly all kernel module developers, 
> etc, by changing internal APIs but this is completely out of hand.

It was marked as obsoleted for some time ... I guess marking something
'osboleted' means that _NO_ new project should depends on it, and also
existing projects should be ported to the newer solutions. The purpose of
this process is to leave enough time for developers to react. I can't see
any problem here. You would be right if devfs would have been removed some
day without any notice before.
 
> Normally I wouldn't care, and I'd just stay away from 'stable' until 
> someone finally figured out that a dev tree really is needed, but I can't 
> stay quiet anymore.  2.6.x is anything but stable right now.  It might be 
> stable in the sense that most any development kernel is stable in that it 
> runs without crashing, but it's not at all stable in the sense that 
> everything is changing as if it were an odd numbered dev tree.

Ah, I see your point. But is it really a BIG problem? I mean please mention
some *real* issue/story confirm your opinion. Sure, you can find, but also
compare it with the advantages of new development model, since there is nothing
in the world which is only have advantages neither something which only has
disadvantages ... The would is not black or white, but a great spectrum of
gray shades.

> Yes, I'm venting some frustrations here, but I can't be the only one.  I 
> know now I'm going to be called a troll or a naysayer but whatever.  The 
> fact is it needs saying.  I shouldn't have to do major changes to 
> accomodate sysfs in a *STABLE* kernel when going between point revs.  This 
> is just not how it's been done in the past.

sysfs should not used by an average application, I guess, so it's not a major
point

-- 
- Gábor

Re: Development tree, PLEASE?

[sean]

On Fri, 20 Jan 2006 09:36:35 -0700
Michael Loftis <mloftis@wgops.com> wrote:

> Yes I realise this change isn't out of the blue or anything, but it's in a 
> 'stable' kernel.  Why bother calling 2.6 stable?  We may as well have 
> stayed at 2.5 if this sort of thing is going to continue to be pulled.
> 

Most of your message seems to boil down to complaining that devfs has been removed.
Unfortunately your frustration is pointed in the wrong direction; you should be
asking yourself why you ignored all those warnings about devfs removal for so long.
If you really need it, just use the 2.4 kernel; it's very stable.

If you want to complain about the current tree being called "stable", then just
don't call it stable.   Call it the development tree because in the end that's what 
it is.  No need to get hung up on a name; it is what it is.  Anyone, is free to fork
a real stable tree just like some distributions do.   But such "stable" trees just 
aren't going to be maintained by the same people who develop the mainline; they have
enough to do already.

If you can think of an idea to solve your problem without demanding that other people 
(ie. the mainline developers) do extra work, then speak up.   But just demanding that
the developers patch a stable tree while working on the development branch as well,
has other _real_ costs that precipitated the shift to the current model.

Sean

Re: Development tree, PLEASE?

[Diego Calleja]

El Fri, 20 Jan 2006 09:36:35 -0700,
Michael Loftis <mloftis@wgops.com> escribió:


> That's the nail on the head exactly.  Why is this being done in an even 
> numbered kernel?  This represents an API change that has knock on well 
> outside of the kernel, and should be done in development releases.  Why is 
> it LK is the only major project (that I know of) that does this?  This is 
> akin to apache changing the format of httpd.conf and saying in say 1.3.38 
> and saying 'well we made the userland tools too.'

There's a Documentation/feature-removal-schedule.txt file. Is not that devfs
and other features were removed suddenly from one day to another. If external
developers don't care about maintaining code in (say) a 6 month timeframe
kernel developers can't do nothing. External developers are encouraged to
merge their code in the main tree anyway.

It's strange that you mention the devfs case. People wanted to remove 
devfs one or maybe two years ago, and Linus and/or akpm decided to kept
it to give people time to migrate. 

Please see the archives to understand why the people who maintains the
kernel and gets their ass kicked when a stable released has a bug decided 
to set up this development model.

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 20, 2006 5:41:13 PM +0100 Jan Engelhardt 
<jengelh@linux01.gwdg.de> wrote:

>
>> Lots of things still out there depend on devfs.  So now if I want to
>> develop my kmod on recent kernels I have to be in the business of
>> maintaining a lot more userland stuff, like mkinitrd, installers, etc.
>> that have come to rely on devfs.
>
> Just like the OSS-ALSA/e100 debate: If there IS something that you
> do not like [something that requires devfs], why has NO ONE objected?
> (Quoting Greg: "and I have not heard a single peep out of anyone about
> the  email titled "Subject: devfs going away, last chance to complain"")
> Not to  forget there is ndevfs if you really need it.

Unfortunately I wasn't pushing on bleeding edge kernels when that thread 
happened and I apologize for not speaking up then, but this is much larger 
than just devfs.  This is the need for development to get off the stable 
kernel, and onto a development branch where it belongs so we can quit 
breaking things for the stable kernel.  Unless the intent is to just not 
have a stable kernel anymore.  If it is, then fine, lets see word from the 
forces that be along those lines.

I'm just calling things as they are, right now, 2.6 is development, and 
unstable.  Yes it runs without crashing and is stable in that sense, but so 
are a goodly lot/most of the 2.5 series (heck I ran a decent chunk of that 
in production).

The problem here is I'm spending a lot of my time lately fixing things that 
shouldn't need fixing.  Things that are/were developed against what was 
supposed to be a stable major version and has been turned into a 
development version.

I realise that there are/have been policy changes, and I can see the need 
and reason behind those, and I agree with them on the front, more 
development is good.  But it should be done in a development branch, 
because otherwise it makes it damn near impossible to maintain when the 
world is slowly changing under you.

It's easier for an embedded system especially to pick a target, and then 
stay with it.  Later when a new major version comes out the time can then 
be invested ONCE to redevelop what needs redeveloping, which is easier to 
do (yes I'm speaking from a business standpoint, sorry, but someone has to) 
and to sell to management than nickel-and-dime to death of trying to follow 
a development tree.

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 20, 2006 11:50:32 AM -0500 Dmitry Torokhov 
<dmitry.torokhov@gmail.com> wrote:

> On 1/20/06, Michael Loftis <mloftis@wgops.com> wrote:
>>
>> Well there's a whole grab bag of them that I'll be getting to over the
>> next few months, but the most immediate is the fact that I've gotten new
>> hardware from a venduh that requires me to build a new Debian installer
>> and new debian kernels.  I also have custom packages that depend on
>> devfs being there and now it's not.
>>
>> Yes I realise this change isn't out of the blue or anything, but it's in
>> a 'stable' kernel.  Why bother calling 2.6 stable?  We may as well have
>> stayed at 2.5 if this sort of thing is going to continue to be pulled.
>>
>
> Ok, so you agree that there was an ample warning that devfs is going
> away... Now, what would be different if 2.8.0 released tomorrow
> without devfs and your vendor would require you to build new Debian
> installer and kernel?

Because that would be expected.  That constitutes a major release, and 
should theoretically have had a development tree corresponding before it.

The problem is changing APIs mid-stream in what's being (or atleast been) 
called a stable release.

I'm trying to think of a way to relate this better but I just can't. 
What's needed is a 'target' for incremental updates, things like minor 
changes, bugfixes, etc.  I feel like supporting entirely new hardware 
(provided that the API is 'frozen' when it's realeased mainstream) in a 
stable kernel is fine, even adding features and functionality to existing 
stuff is fine but without having to take the whole damned enchilada of 
changes a development tree entails, which is all of the internal APIs. 
Yeah, it would/will become generally stagnant tree, but that's the point of 
a stable release.

It's horrificly expensive to maintain large numbers of machines (even if 
it's automated) as it is.  If you're doing embedded development too or 
instead, it gets even harder when you need certain bugfixes or minor 
changes, but end up having to redevelop things or start maintaining your 
own kernel fork.

I know there won't be any change today, or right now, or maybe not ever, 
but it seems to me that this is a step backwards in lk development. 
Without a stable release for vendors and distros, and yes, commercial ones 
at that, they either have to spend a LOT more time trying to keep track of 
all the changes and pushing them out constantly to their clients and users, 
it discourages development in that area, makes them go elsewhere where they 
don't have to deal with this.

I'm hearing a lot from developers of embedded type systems about this 
recent change in development.  Though, of those, I'm probably the only one 
who even occasionally pops up on LKML here.

Hopefully this last bit won't get buried in noise....

I really understand atleast some of the reasons from the kernel development 
standpoint, and can see many really good reasons for running a development 
tree like this, and as a method of development I like and agree with it. 
However...for the general consumption there still needs to be some sort of 
stable target that can be used that's 'blessed' with that mark, and will 
get atleast some attention by developers for security updates and (mostly 
major) bugfixes, and that people can contribute these sorts of things to, 
probably with the proviso that they also contribute it to the mainline dev 
kernel maybe IE if you're going to add new supported device to 'stable' 
2.6.16.x then you've got to add it to whatever the current 'dev' line is 
say 2.6.22 or whatever.  The placing of the .'s is just symbolic.  It could 
be 2.6.x and 2.7.x just as in the past or it could be 3.0.0.x and 3.0.0+n

Re: Development tree, PLEASE?

[Diego Calleja]

El Fri, 20 Jan 2006 10:06:50 -0700,
Michael Loftis <mloftis@wgops.com> escribió:

> The one machine I've got running 2.6+devfs under debian chokes in initrd 
> with an inability to find devfs during boot so I had to go back to static 
> /dev entries for it since atleast in sarge right now I'm not seeing a 

Were you using a debian-provided kernel? 

> quick-and-easy way to get devfs like support bundled via mkinitrd, but I 
> haven't looked, and I shouldn't have to.  It shouldn't have gone away in a 

Have you updated other packages in your system? If you update the kernel, 
you should update the rest of the kernel-related userspace tools (mkinitrd 
is hardly "yet another userspace application"). Debian default kernel
is 2.4 and only provides a 2.6.8-based kernel (2.6.8 was released in august
2004, 17 months ago) as an alternative, and the debian guys put this
in the release notes:

http://www.debian.org/releases/stable/i386/release-notes/ch-information.en.html#s-upgrade-to-2.6

"[...] Upgrading to a 2.6 kernel from an earlier version is therefore
not a process to be undertaken lightly" and "5.2.4 Switching to 2.6 may
activate udev"

Re: Development tree, PLEASE?

[Joe George]

Randy.Dunlap wrote:
> On Fri, 20 Jan 2006, Joe George wrote:
> 
>> Michael Loftis wrote:
>>>
>>> --On January 20, 2006 4:29:44 PM +0000 James Courtier-Dutton
>>> <James@superbug.co.uk> wrote:
>>>
>>>> It is unclear what you are really ranting about here. The "stable" kernel
>>>> is stable or at least as stable as it is going to be. It is left to
>>>> distros to make it even more stable. The interface to user land has not
>>>> changed.
>>>> If all you are ranting about is the move from devfs to udevd, then all
>>>> the user land tools dealing with them have been updated already.
>>> That's the nail on the head exactly.  Why is this being done in an even
>>> numbered kernel?  This represents an API change that has knock on well
>>> outside of the kernel, and should be done in development releases.  Why
>>> is it LK is the only major project (that I know of) that does this?
>>> This is akin to apache changing the format of httpd.conf and saying in
>>> say 1.3.38 and saying 'well we made the userland tools too.'
>>>
>>>> What is the real specific problem you are having?
>>> Well there's a whole grab bag of them that I'll be getting to over the
>>> next few months, but the most immediate is the fact that I've gotten new
>>> hardware from a venduh that requires me to build a new Debian installer
>>> and new debian kernels.  I also have custom packages that depend on
>>> devfs being there and now it's not.
>>>
>>> Yes I realise this change isn't out of the blue or anything, but it's in
>>> a 'stable' kernel.  Why bother calling 2.6 stable?  We may as well have
>>> stayed at 2.5 if this sort of thing is going to continue to be pulled.
>>>
>> I don't think that kernel developers are calling 2.6 a stable kernel
>> series.  There was an evolution into another development model without
>> a corresponding change in the kernel numbering.  I think the main
>> reason the numbering wasn't changed was that it would break thousands
>> of scripts people are using all over the world.
>>
>> What would be nice is to go, for example, from 2.6.17 to 3.1, 3.2,
>> 3.3, ...  And have what is currently called the stable series start at
>> 3.1.1.  This would make it clear that the 2.4/2.5 way of doing business
>> is over.  Someone would have to decide whether it is worth it to break
>> all the scripts, however.
> 
> The problems AFAICT are:
> 
> 1.  We did (for 2.5/2.4) or would (for 3.3/3.2) spend tons of time
> in backporting new features or drivers from the development tree
> to the stable tree.  The current model saves that duplication
> (or even worse if multiple distros do that same work).
> 
> 2.  If we did have a separate development tree, we would need
> to clone Andrew. 8:)  IMO there aren't a lot of choices for qualified
> tree maintainers, although I'm sure we could find someone if we
> had to.
> 
> Anyway, to summarize, it's about manpower and efficient use of it.
> 

I agree with all that and I would not want to change the way things
work at all.  I just wish that the number could be changed so the
rest of the world would realize it changed and wouldn't keep saying
2.6 is a stable series.

Joe

Re: Development tree, PLEASE?

[Dmitry Torokhov]

On 1/20/06, Marc Koschewski <marc@osknowledge.org> wrote:
> * Dmitry Torokhov <dmitry.torokhov@gmail.com> [2006-01-20 11:55:38 -0500]:
>
> > On 1/20/06, Marc Koschewski <marc@osknowledge.org> wrote:
> > > * Dmitry Torokhov <dmitry.torokhov@gmail.com> [2006-01-20 > >
> > > > Marc, have you tried 2.6.16-rc1 yet? Does it also give you problems
> > > > with psmouse?
> > > >
> > >
> > > Not yet, Dmitry. I just managed to get today's -git compiled and running. I'll
> > > give it a try tonite.
> > >
> > > Moreover, I put a
> > >
> > >        echo -n 0 > /sys/bus/serio/devices/serio0/resync_time
> > >
> > > in my initscripts. Since then I didn't see any problem. I'll report later how it
> > > went with that line removed and stock 2.6.16-rc1.
> > >
> >
> > Ok, if psmouse gives you a fit and setting resync_time to 0 cures it,
> > please do the follwing:
> >
> > echo -n 5 > /sys/bus/serio/devices/serioX/resync_time
> > echo 1 > /sys/modules/i8042/parameters/debug
> > ... wait 10 seconds ...
> > move the mouse slightly
> > ... wait another 10 seconds ...
> > move the mouse slighty again
> > echo 0 > /sys/modules/i8042/parameters/debug
> >
> > and send me your dmesg (or better /var/log/messages or whatever file
> > you use for kernel messages).
> >
> > --
> > Dmitry
>
> OK, here goes:
>

Hmm, I see 2 perfectly healthy resyncs. Could you remind me what is
exactly wrong with the mouse on your box? Does it give you fits right
now (you seem to leave resync on)?

--
Dmitry

Re: Development tree, PLEASE?

[Marc Koschewski]

* Dmitry Torokhov <dmitry.torokhov@gmail.com> [2006-01-20 12:43:12 -0500]:

> On 1/20/06, Marc Koschewski <marc@osknowledge.org> wrote:
> > * Dmitry Torokhov <dmitry.torokhov@gmail.com> [2006-01-20 11:55:38 -0500]:
> >
> > > On 1/20/06, Marc Koschewski <marc@osknowledge.org> wrote:
> > > > * Dmitry Torokhov <dmitry.torokhov@gmail.com> [2006-01-20 > >
> > > > > Marc, have you tried 2.6.16-rc1 yet? Does it also give you problems
> > > > > with psmouse?
> > > > >
> > > >
> > > > Not yet, Dmitry. I just managed to get today's -git compiled and running. I'll
> > > > give it a try tonite.
> > > >
> > > > Moreover, I put a
> > > >
> > > >        echo -n 0 > /sys/bus/serio/devices/serio0/resync_time
> > > >
> > > > in my initscripts. Since then I didn't see any problem. I'll report later how it
> > > > went with that line removed and stock 2.6.16-rc1.
> > > >
> > >
> > > Ok, if psmouse gives you a fit and setting resync_time to 0 cures it,
> > > please do the follwing:
> > >
> > > echo -n 5 > /sys/bus/serio/devices/serioX/resync_time
> > > echo 1 > /sys/modules/i8042/parameters/debug
> > > ... wait 10 seconds ...
> > > move the mouse slightly
> > > ... wait another 10 seconds ...
> > > move the mouse slighty again
> > > echo 0 > /sys/modules/i8042/parameters/debug
> > >
> > > and send me your dmesg (or better /var/log/messages or whatever file
> > > you use for kernel messages).
> > >
> > > --
> > > Dmitry
> >
> > OK, here goes:
> >
> 
> Hmm, I see 2 perfectly healthy resyncs. Could you remind me what is
> exactly wrong with the mouse on your box? Does it give you fits right
> now (you seem to leave resync on)?

Well, the pointer seems to be very happy when jumping into the (mostly) upper
right corner. Then it seems like movement and clicks somehow get swallowed
away or stacked and after that get issued in a) either wrong order or b) wrong.
Moreover, even if I didn't click any button (including btn 4 + 5 for wheel
up/down) the mouse clicks into the window what often opens context menus.
Sometimes it then even clicks in. Once it logged me off that way from GNOME
because selecting the entry from the menu and hit 'Log out'. Summary: unusable.

Let me repeat this with a clean 2.6.16-rc1 install without any patches and
resync_timing of 5 tonite. I'll send the whole kern.log part from gdm login (the
agpgart lines) until the mouse jumps then. 

Marc

Re: Development tree, please?

[Michael Loftis]

(I changed the please to a lower case, I was overly punchy in the subject 
line, and I apologize to everyone for that)

--On January 20, 2006 12:11:16 PM -0500 sean <seanlkml@sympatico.ca> wrote:

> On Fri, 20 Jan 2006 09:36:35 -0700
> Michael Loftis <mloftis@wgops.com> wrote:
>
>> Yes I realise this change isn't out of the blue or anything, but it's in
>> a  'stable' kernel.  Why bother calling 2.6 stable?  We may as well have
>> stayed at 2.5 if this sort of thing is going to continue to be pulled.
>>
>
> Most of your message seems to boil down to complaining that devfs has
> been removed. Unfortunately your frustration is pointed in the wrong
> direction; you should be asking yourself why you ignored all those
> warnings about devfs removal for so long. If you really need it, just use
> the 2.4 kernel; it's very stable.

It is.  And the majority of the systems I've built (and still most new 
installs) use 2.4, but because of the need in many situations for things 
that only existed starting in 2.5 there's more argument for many cases for 
2.6 (and with some of the ARM development I've got going on there's even 
more argument for 2.6...despite the headers playing games with me right 
now....)

> If you want to complain about the current tree being called "stable",
> then just don't call it stable.   Call it the development tree because in
> the end that's what  it is.  No need to get hung up on a name; it is what
> it is.  Anyone, is free to fork a real stable tree just like some
> distributions do.   But such "stable" trees just  aren't going to be
> maintained by the same people who develop the mainline; they have enough
> to do already.

I was under the impression that the consensus has usually been multiple 
forks dividing a lot of external development resources into their own 
little camps instead of letting them all contribute to the main kernel was 
considered a bad thing?  Has this changed?  I know it's better for the 
developers....but shouldn't they or...'someone' be responsible for 
maintenance and have a place to do so?  Community maintenance? 
Developer+maintainer pairs in cases where the developer is unable or 
unwilling to actually maintain his/her code?

A target for such efforts, and community support for them would continue 
the ... 'tradition' of this being a community kernel where efforts are 
focused, and not where efforts are turned away and told to maintain your 
own forks.

>
> If you can think of an idea to solve your problem without demanding that
> other people  (ie. the mainline developers) do extra work, then speak up.
> But just demanding that the developers patch a stable tree while working
> on the development branch as well, has other _real_ costs that
> precipitated the shift to the current model.

Having a stable tree would atleast give me a place to commit changes to 
publicly where/if I needed to.  My main concern *today* is the devfs 
problems which I can solve yes and were known about yes, but require quite 
a bit of effort just to support the second problem of *today* which is 
Intel's latest e1000 variant.  That though is just today's troubles right 
now.  I can see others coming, and I'm concerned.

I understand the reason, but the problem it creates is it does give a lot 
of places incentive to just not contribute their bugfixes, and instead fork 
since they're not interested in getting involved in API changes 'right now'.

Re: Development tree, PLEASE?

[Dmitry Torokhov]

On 1/20/06, Marc Koschewski <marc@osknowledge.org> wrote:
>
> Well, the pointer seems to be very happy when jumping into the (mostly) upper
> right corner. Then it seems like movement and clicks somehow get swallowed
> away or stacked and after that get issued in a) either wrong order or b) wrong.
> Moreover, even if I didn't click any button (including btn 4 + 5 for wheel
> up/down) the mouse clicks into the window what often opens context menus.
> Sometimes it then even clicks in. Once it logged me off that way from GNOME
> because selecting the entry from the menu and hit 'Log out'. Summary: unusable.

Ok, I remember now. Andall this wierdness goes away with resync_time
set to 0, right?

>
> Let me repeat this with a clean 2.6.16-rc1 install without any patches and
> resync_timing of 5 tonite. I'll send the whole kern.log part from gdm login (the
> agpgart lines) until the mouse jumps then.
>

Could you please note exact time when mouse jumps - this will help me
locate the spot in the log.

--
Dmitry

Re: Development tree, PLEASE?

[Marc Koschewski]

* Dmitry Torokhov <dmitry.torokhov@gmail.com> [2006-01-20 13:00:16 -0500]:

> On 1/20/06, Marc Koschewski <marc@osknowledge.org> wrote:
> >
> > Well, the pointer seems to be very happy when jumping into the (mostly) upper
> > right corner. Then it seems like movement and clicks somehow get swallowed
> > away or stacked and after that get issued in a) either wrong order or b) wrong.
> > Moreover, even if I didn't click any button (including btn 4 + 5 for wheel
> > up/down) the mouse clicks into the window what often opens context menus.
> > Sometimes it then even clicks in. Once it logged me off that way from GNOME
> > because selecting the entry from the menu and hit 'Log out'. Summary: unusable.
> 
> Ok, I remember now. Andall this wierdness goes away with resync_time
> set to 0, right?

Yes, correct.

> 
> >
> > Let me repeat this with a clean 2.6.16-rc1 install without any patches and
> > resync_timing of 5 tonite. I'll send the whole kern.log part from gdm login (the
> > agpgart lines) until the mouse jumps then.
> >
> 
> Could you please note exact time when mouse jumps - this will help me
> locate the spot in the log.

I will. 

Re: Development tree, please?

[sean]

On Fri, 20 Jan 2006 10:56:25 -0700
Michael Loftis <mloftis@wgops.com> wrote:


> It is.  And the majority of the systems I've built (and still most new 
> installs) use 2.4, but because of the need in many situations for things 
> that only existed starting in 2.5 there's more argument for many cases for 
> 2.6 (and with some of the ARM development I've got going on there's even 
> more argument for 2.6...despite the headers playing games with me right 
> now....)

You see, that's exactly the problem.   Say you have a a mainline "stable" 
tree called 2.8 which still had devfs in it, and a development branch 2.9
with it removed.   If you end up needing something new in the 2.9 branch 
where devfs is remvoed you're in _exactly_  the same situation you find 
yourself in now.   

Essentially what you're saying is you need some stuff from the development 
branch (ie. why 2.4 is unacceptable to you today) but you want to pick 
and choose which pieces.

When you need some of the pieces that are only in the latest mainline
kernels you just _have_ to accept that other things will have changed
as well.   Changing the names of things isn't going to change that one
bit.


> I was under the impression that the consensus has usually been multiple 
> forks dividing a lot of external development resources into their own 
> little camps instead of letting them all contribute to the main kernel was 
> considered a bad thing?  Has this changed?  I know it's better for the 
> developers....but shouldn't they or...'someone' be responsible for 
> maintenance and have a place to do so?  Community maintenance? 
> Developer+maintainer pairs in cases where the developer is unable or 
> unwilling to actually maintain his/her code?
> 
> A target for such efforts, and community support for them would continue 
> the ... 'tradition' of this being a community kernel where efforts are 
> focused, and not where efforts are turned away and told to maintain your 
> own forks.

Well you do have a point there, but the counter point still remains.   The
current mainline developers are just too busy to fill this role.   There 
are costs associated with any model and the current model at least
reduces the costs borne by the mainline developers.   It would be nice
if someone would step up and provide a central repository for a stable
branch; but who?   I really don't know, but complaining to the current
mainline developers is the wrong approach to solving the issue.

> 
> Having a stable tree would atleast give me a place to commit changes to 
> publicly where/if I needed to.  My main concern *today* is the devfs 
> problems which I can solve yes and were known about yes, but require quite 
> a bit of effort just to support the second problem of *today* which is 
> Intel's latest e1000 variant.  That though is just today's troubles right 
> now.  I can see others coming, and I'm concerned.

There's no doubt that there are upsides to a mainline stable tree.    The
point is that there are also _costs_.   _You_ have to suggest a solution
that would offer a stable mainline tree and not cost the current mainline
developers anything.

> 
> I understand the reason, but the problem it creates is it does give a lot 
> of places incentive to just not contribute their bugfixes, and instead fork 
> since they're not interested in getting involved in API changes 'right now'.
> 

Yes.   It's all about tradeoffs.   The current model spreads the costs out
to more people than just the mainline developers.   In the end it's more 
fair than the older model and actually allows the developers to provide
us all with a better cutting edge kernel faster.   No doubt that there
are some downsides, but you haven't offered a way to pay for the costs
associated with going back to the old model.  

Sean

Re: Development tree, please?

[Michael Loftis]

--On January 20, 2006 1:11:20 PM -0500 sean <seanlkml@sympatico.ca> wrote:

> You see, that's exactly the problem.   Say you have a a mainline "stable"
> tree called 2.8 which still had devfs in it, and a development branch 2.9
> with it removed.   If you end up needing something new in the 2.9 branch
> where devfs is remvoed you're in _exactly_  the same situation you find
> yourself in now.

Except for there's atleast an old stable branch, as there has been in the 
past, and the change was made and having to track the NEW stable branch. 
Where do people apply security updates and patches to now that there's no 
stable branch out there?  Or is that just discouraged now?

> Essentially what you're saying is you need some stuff from the
> development  branch (ie. why 2.4 is unacceptable to you today) but you
> want to pick  and choose which pieces.

Negative, my big problem is the lack of a stable branch.  2.4 is really 
showing it's age in many places.  Lack of hardware support, lack of a lot 
of networking improvments in 2.6, lack of SMP improvments in 2.6....the 
list is pretty good.  If 2.6 isn't stable then why the 2.5->2.6 at all. 
Why isn't this still 2.5?  because people want/needed a new stable rev.

I know there will *always* be problems and issues with picking any point in 
time, arbitrary or not, to say lets work up for a feature freeze for 
release.  The Debian project knows this all to well.  However the quality 
of their releases has been so much better than anyone elses, I still stick 
with them as much as I can.

> When you need some of the pieces that are only in the latest mainline
> kernels you just _have_ to accept that other things will have changed
> as well.   Changing the names of things isn't going to change that one
> bit.

OK so I should have to change all my userland utilities, installers, etc 
just because I need a new PCI ID?  Or just because there's a *HUGE* bug in 
say, aic7xxx?  devfs is just one facet of a potentially bigger problem. 
(which you do seem to understand, so...a lot of this reply is for the 
benefit of the larger audience)


> Well you do have a point there, but the counter point still remains.   The
> current mainline developers are just too busy to fill this role.   There
> are costs associated with any model and the current model at least
> reduces the costs borne by the mainline developers.   It would be nice
> if someone would step up and provide a central repository for a stable
> branch; but who?   I really don't know, but complaining to the current
> mainline developers is the wrong approach to solving the issue.

If someone were to step up with a svn (or git...i'm not familiar with git 
yet though) or similar repository, would it be 'blessed' by the developers 
though, and able to maintain a version stream?  This would give the 
community, and the developers that wished or were able a place to push out 
changes that are of a maintenance nature.  Yes maintenance is a headache, 
but 90% of development is maintenance ;)  atleast in the long view.  I 
don't want, nor think that the developers should 'maintain' indefinitely 
old versions, but having a target for maintenance would be good, assuming 
it was 'blessed' and made to be a common place.

> There's no doubt that there are upsides to a mainline stable tree.    The
> point is that there are also _costs_.   _You_ have to suggest a solution
> that would offer a stable mainline tree and not cost the current mainline
> developers anything.
>
>>
>> I understand the reason, but the problem it creates is it does give a
>> lot  of places incentive to just not contribute their bugfixes, and
>> instead fork  since they're not interested in getting involved in API
>> changes 'right now'.
>>
>
> Yes.   It's all about tradeoffs.   The current model spreads the costs out
> to more people than just the mainline developers.   In the end it's more
> fair than the older model and actually allows the developers to provide
> us all with a better cutting edge kernel faster.   No doubt that there
> are some downsides, but you haven't offered a way to pay for the costs
> associated with going back to the old model.

OK well....we're certainly on the same page....and seem to have an idea of 
where to go.  I guess just how to get there is the problem.

If a community effort was started...and provided we (this proposed 
community effort, which will probably have atleast some overlap with some 
mainline kernel developers) can get the blessing of the mainline effort I 
think it might work.  Especially if one or more distro's would sign on to 
the effort and publish their maintenance and bugfixes there as well.  The 
general idea being that a certain number of releases are picked to maintain 
as maintenance releases, hopefully with feedback from people on bugs and 
preferably with feedback of patches and/or commits to fix bugs and maintain 
a 'stable' branch that has atleast some hope of having a larger than one 
investment in keeping security updates atleast, and minor enhancements. 
There'd be someone ultimately responsible for it all, just as there is for 
the mainline, to coordinate and make releases based on that.  Likely these 
branches/forks would be pretty quiet just fixing mainline bugs and minor 
things that are needed in the course of normal maintenance of a project.

>
> Sean
>



--
"Genius might be described as a supreme capacity for getting its possessors
into trouble of all kinds."
-- Samuel Butler

Re: Development tree, PLEASE?

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 740 bytes --]

On Fri, 20 Jan 2006 10:31:12 MST, Michael Loftis said:

> It's horrificly expensive to maintain large numbers of machines (even if 
> it's automated) as it is.  If you're doing embedded development too or 
> instead, it gets even harder when you need certain bugfixes or minor 
> changes, but end up having to redevelop things or start maintaining your 
> own kernel fork.

But you're perfectly happy to make the kernel developers do the equivalent thing
when they have to maintain 2 forks (a stable and devel).  Go back and look at
the status of the 2.5 tree - there were *large* chunks of time when 2.4 or 2.5
would get an important bugfix, but the other tree wouldn't get it for *weeks*
because of the hassle of cross-porting the patch.

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 20, 2006 2:03:52 PM -0500 Valdis.Kletnieks@vt.edu wrote:

> But you're perfectly happy to make the kernel developers do the
> equivalent thing when they have to maintain 2 forks (a stable and devel).
> Go back and look at the status of the 2.5 tree - there were *large*
> chunks of time when 2.4 or 2.5 would get an important bugfix, but the
> other tree wouldn't get it for *weeks* because of the hassle of
> cross-porting the patch.

Weeks is better than never, and still better than commercial vendors. ;)

--
"Genius might be described as a supreme capacity for getting its possessors
into trouble of all kinds."
-- Samuel Butler

Re: Development tree, PLEASE?

[Greg KH]

On Fri, Jan 20, 2006 at 08:17:40AM -0700, Michael Loftis wrote:
> 
> Normally I wouldn't care, and I'd just stay away from 'stable' until 
> someone finally figured out that a dev tree really is needed, but I can't 
> stay quiet anymore.  2.6.x is anything but stable right now.  It might be 
> stable in the sense that most any development kernel is stable in that it 
> runs without crashing, but it's not at all stable in the sense that 
> everything is changing as if it were an odd numbered dev tree.

Please detail your specific problems that you are having.  General rants
don't help anyone except make you feel better :)

thanks,

greg k-h

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 20, 2006 2:03:52 PM -0500 Valdis.Kletnieks@vt.edu wrote:

> But you're perfectly happy to make the kernel developers do the
> equivalent thing when they have to maintain 2 forks (a stable and devel).
> Go back and look at the status of the 2.5 tree - there were *large*
> chunks of time when 2.4 or 2.5 would get an important bugfix, but the
> other tree wouldn't get it for *weeks* because of the hassle of
> cross-porting the patch.

To more fully respond though....

Weeks is fine, and better than never.  And there may be cases in which the 
decision has to be made to 'abandon' a particular stable release in favor 
of a newer version because of the difficulty or inability to backport fixes.

I think that it's fine to push the maintenance effort away from the 
mainline developers, probably even desireable, but then the bugfixing/etc 
tends to happen in a disparate manner, off on lots of forks at different 
places without them making their way back to some central place.

And that seems where we're going with this conversation.  A fork/forks at 
various versions to maintain bugfixes and support updates that's (more?) 
open to submitters writing patches.  Maintained by a separate group or 
party, but with the 'blessing' from mainline to do so.  A place for those 
sorts of efforts to be focused, without necessarily involving the primary 
developers.

Re: Development tree, PLEASE?

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 504 bytes --]

On Fri, 20 Jan 2006 12:21:40 MST, Michael Loftis said:

> And that seems where we're going with this conversation.  A fork/forks at 
> various versions to maintain bugfixes and support updates that's (more?) 
> open to submitters writing patches.  Maintained by a separate group or 
> party, but with the 'blessing' from mainline to do so.  A place for those 
> sorts of efforts to be focused, without necessarily involving the primary 
> developers.

Congrats. You just re-invented bugzilla.redhat.com.

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Development tree, PLEASE?

[Ben Collins]

On Fri, 2006-01-20 at 08:17 -0700, Michael Loftis wrote:
> OK, I don't know abotu others, but I'm starting to get sick of this 
> unstable stable kernel.  Either change the statements allover that were 
> made that even-numbered kernels were going to be stable or open 2.7. 
> Removing devfs has profound effects on userland.  It's one thing to screw 
> with all of the embedded developers, nearly all kernel module developers, 
> etc, by changing internal APIs but this is completely out of hand.

Me, personally, I like it. It's much easier for the distro maintainers
in that they can get the latest and greatest stuff without waiting an
entire year for 2.<odd>.x to turn into 2.<even>.0, and wait a little
more until 2.<even>.0 becomes something like 2.<even>.8 for it to be
stable (because no one was testing the millions of lines of new code
going into the development branch).

I think the new model is also easier for new
drivers/filesystems/whatever, since they don't have to wait for the next
development 2.<odd> branch to get their code in, and then wait for
2.<even>.0 to be released so normal users and distros will get their new
feature.

It also keeps development moving along _very_ quickly, and reduces how
stale the stable kernel tree becomes. When 2.5.0 started, developers
stopped working on 2.4.x because it was just too damn much work to track
two trees. So 2.4.x became stagnant, and while development was moving on
2.5.x, no one other than hardcore developers were using it, so there was
very little testing of a tree that was getting a years worth of code
changes.

So put me in for +1 on the current development model. It suits me,
Ubuntu, and the Ubuntu users very well. We are on a 6 month release
cycle, so a new kernel every 3 months fits us perfect. Means every 6
months we can release a 3 month old kernel. Just old enough to be
stable, not so old it's useless.

-- 
Ben Collins
Kernel Developer - Ubuntu Linux

Re: Development tree, PLEASE?

[Greg KH]

On Fri, Jan 20, 2006 at 10:14:15AM -0700, Michael Loftis wrote:
> The problem here is I'm spending a lot of my time lately fixing things that 
> shouldn't need fixing.  Things that are/were developed against what was 
> supposed to be a stable major version and has been turned into a 
> development version.

What specifically are you "fixing"?

What version are you finding things that work, and then later, not work?

Are you properly looking at the required version of things in the README
file?

Again, specifics please, otherwise nothing can have a chance of getting
better.

thanks,

greg k-h

Re: Development tree, PLEASE?

[Russell King]

On Fri, Jan 20, 2006 at 12:21:40PM -0700, Michael Loftis wrote:
> I think that it's fine to push the maintenance effort away from the 
> mainline developers, probably even desireable, but then the bugfixing/etc 
> tends to happen in a disparate manner, off on lots of forks at different 
> places without them making their way back to some central place.

The responsibility for ensuring that those bugfixes get back to "some
central place" is with the folk who created the bugfixes.

I've seen this _far_ too many times - it's what I call "the CVS disease"
and it happens _a_ _lot_ in certain areas.

Developer X uses a CVS tree for his work and has write access to that
tree.  He finds a bug, and fixes it.  Maybe he writes a good explaination
of the bug and puts it in the CVS commit comments.  He commits the fix.
When he's done with development, he walks away and the fix never gets
submitted.  (Not everyone operates this way, but I know some do.)


As a mainline guy myself, I'll say we're all welcome to whatever bug
fixes come our way.  We just need someone to send them to us with an
adequate explaination of the bug.

> And that seems where we're going with this conversation.  A fork/forks at 
> various versions to maintain bugfixes and support updates that's (more?) 
> open to submitters writing patches.  Maintained by a separate group or 
> party, but with the 'blessing' from mainline to do so.  A place for those 
> sorts of efforts to be focused, without necessarily involving the primary 
> developers.

Do you know about the bugfix-only kernel series, which have 4-digit
versions - 2.6.x.y - which is the compromise to satisfy folk with the
issue you're bringing up in this paragraph?

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Re: Development tree, PLEASE?

[James Courtier-Dutton]

Michael Loftis wrote:
> To more fully respond though....
> 
> Weeks is fine, and better than never.  And there may be cases in which 
> the decision has to be made to 'abandon' a particular stable release in 
> favor of a newer version because of the difficulty or inability to 
> backport fixes.
> 
> I think that it's fine to push the maintenance effort away from the 
> mainline developers, probably even desireable, but then the 
> bugfixing/etc tends to happen in a disparate manner, off on lots of 
> forks at different places without them making their way back to some 
> central place.
> 

Please be more specific or stop posting. No one is going to bother 
answering you with any useful answer unless you are more specific.

 From the general trend of the thread so far, it seems like a fairly 
pointless request to me.

All well known distros had absolutely no problem with devfs -> udevd 
transition, so why are you having problems?

If you stop spending time posting, you might actually have time to fix 
your problems.

James

Re: Development tree, PLEASE?

[Jesper Juhl]

On 1/20/06, Michael Loftis <mloftis@wgops.com> wrote:
>
[snip]
> I'm trying to think of a way to relate this better but I just can't.
> What's needed is a 'target' for incremental updates, things like minor
> changes, bugfixes, etc.  I feel like supporting entirely new hardware

That's called a vendor kernel.
You pay the vendor money, the vendor maintains a stable (as in feature
frozen) kernel, backports bugfixes for you etc.
Take a look at the RedHat and SuSE enterprise kernels, they seem to be
what you want.

> (provided that the API is 'frozen' when it's realeased mainstream) in a
> stable kernel is fine, even adding features and functionality to existing
> stuff is fine but without having to take the whole damned enchilada of
> changes a development tree entails, which is all of the internal APIs.
> Yeah, it would/will become generally stagnant tree, but that's the point of
> a stable release.
>

In my book 'stable' means 'doesn't crash' and 'doesn't break userspace
without long advance notice', it doesn't mean 'does not evolve/goes
stale'.
And in my oppinion the current 2.6 tree succeeds in being a stable kernel.

Besides, I don't agree with your view that we break userspace all the
time as you seem to be saying in several of your mails, quite the
opposite - a lot of work goes into *not* breaking userspace.
Just take a look at how syscalls are maintained, even old obsolete
ones stay in place since removing them would break userspace. Stuff in
/proc does not get changed since that would potentially break
userspace. Look at the fact that you can still run ancient a.out
binaries on a recent 2.6.x kernel.
Even internal kernel APIs usually stay around with __deprecated or as
wrappers around new APIs for extensive periods of time.
And when stuff is removed it's announced for ages in advance. That
devfs would be removed was announced several *years* before it
actually happened. That old OSS drivers will be removed (but only for
hardware that has ALSA equivalents) has been announced months ago and
the removal won't happen for several months (at least) yet.

I'd say the kernel tries damn hard at maintaining backwards
compatibility for userspace.
It tries less hard, but still makes a pretty good effort, for internal
APIs, but the real solution to the internal API churn is to get your
code merged as it'll then get updated automagically whenever something
changes - people who remove or change internals try very hard to also
update all (in-tree) users of the old stuff - take a look at when I
removed a small thing like verify_area() if you want an example.


> It's horrificly expensive to maintain large numbers of machines (even if
> it's automated) as it is.  If you're doing embedded development too or
> instead, it gets even harder when you need certain bugfixes or minor
> changes, but end up having to redevelop things or start maintaining your
> own kernel fork.
>
The solution here is to get your code merged in mainline.

[snip]

--
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: Development tree, PLEASE?

[Russell King]

On Fri, Jan 20, 2006 at 10:14:15AM -0700, Michael Loftis wrote:
> It's easier for an embedded system especially to pick a target, and then 
> stay with it.  Later when a new major version comes out the time can then 
> be invested ONCE to redevelop what needs redeveloping, which is easier to 
> do (yes I'm speaking from a business standpoint, sorry, but someone has to) 
> and to sell to management than nickel-and-dime to death of trying to follow 
> a development tree.

Please note that the current development strategy suits embedded folk.

With the old model, embedded folk could not wait two years for a (eg)
2.5 kernel to become a 2.6 kernel and then "stabilise".  In two years,
the new SoC is already in full-use in multiple applications and folk
have been and long gone developing their products.

So what happens is a massive conflict of interest - do we put
$mass-of-new-features into 2.4 kernels at the risk of destabilising
the current users, or do we push it into 2.5 and wait two or more
years before folk start using that code.

Like it or not, the latter causes a _lot_ of difficulties for a _lot_
of people, so much so that it's an economic disaster unless you're a
large corporation.  The former is also a non-starter because then you
end up with folk complaining that it should be in the development
branch.  It's a no-win situation - you can't satisfy everyone.

So, our current model is a compromise - you get _told_ that things
will change well in advance, and if you choose to ignore those warnings
(or don't respond to them) that's entirely your own problem.

It's like a stick and carrot scenario - see
http://everything2.com/index.pl?node=stick%20and%20carrot

The carrot in this case is the notice about devfs removal.  The
stick is the actual devfs removal.  One's pleasant, the other isn't.
Which you take notice of is entirely your choice.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Re: Development tree, PLEASE?

[Kyle Moffett]

On Jan 20, 2006, at 12:06, Michael Loftis wrote:
> --On January 20, 2006 5:35:51 PM +0100 Marc Koschewski  
> <marc@osknowledge.org> wrote:
>> Moreover, as far as I remember... my devfsd -> udev transsition  
>> went as smooth as a reboot.
>
> The one machine I've got running 2.6+devfs under debian chokes in  
> initrd with an inability to find devfs during boot so I had to go  
> back to static /dev entries for it since atleast in sarge right now  
> I'm not seeing a quick-and-easy way to get devfs like support  
> bundled via mkinitrd, but I haven't looked, and I shouldn't have to.

Guess what, you _don't_ have to.  I have no less than 4 different 2.6  
debian boxes here, all booting the fully modular stock Debian kernels  
from software RAID on SATA or PATA (depends on the box).  Not only  
that, but I can shut down and rearrange those drives to different IDE/ 
SATA ports, then boot and it all still works with consistent /dev  
names (With the exception that I have to bump yaboot into booting  
from a different OpenFirmware path).  If you've customized and  
hardcoded a lot of the boot scripts, I can understand why things  
might be breaking, but the default initrds that the Debian tools  
generate work just fine for me.

Cheers,
Kyle Moffett

--
There is no way to make Linux robust with unreliable memory  
subsystems, sorry.  It would be like trying to make a human more  
robust with an unreliable O2 supply. Memory just has to work.
   -- Andi Kleen

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 20, 2006 11:43:31 AM -0800 Greg KH <greg@kroah.com> wrote:

> On Fri, Jan 20, 2006 at 10:14:15AM -0700, Michael Loftis wrote:
>> The problem here is I'm spending a lot of my time lately fixing things
>> that  shouldn't need fixing.  Things that are/were developed against
>> what was  supposed to be a stable major version and has been turned into
>> a  development version.
>
> What specifically are you "fixing"?

At this point I'm looking at bugs in the aic7xxx driver, it mostly works in 
2.6.8, occasionally locking up my tape subsystem, it's apparently fixed in 
2.6.15 or 2.6.15.1, I need to look closer into that, and backport it 
because of the devfs issue I don't think I can take 2.6.15/2.6.15.1 whole 
hog.  A decent amount of ARM stuff moving around between even just 2.6.11 
and 2.6.13 (admittedly that's a gripe for ARM) making development for that 
port very painful (there's talk of switching to something else because of 
all of this for those projects) -- no specifics on the ARM stuff as I'm not 
the developer directly involved with most that, I'm just doing some PHY 
code, which will eventually be submitted back to the mainstream ARM (still 
in product development) and he's indisposed today/at the moment, I'll try 
to get him to summarize those issues so I can relay them to the list.

As far as fixing there are modules that have been developped to run various 
embedded peripherals that must be reworked to use the newer kernel 
versions, which wouldn't be a problem if there weren't various other fixes 
that were needed which means moving up point revs.  Most of these other 
bugs are external to my work, but they affect my work.  The modules are 
completely isolated from the rest of the kernel though and they're for very 
particular hardware for different clients.

I'm having really weird serial console issues with 2.6.8, occasionally it 
just seems to 'hang' until one or more software flow control start/stops is 
sent.  This though *might* have something also to do with the blade 
chassis' virtual serial stuff, but however it's been totally fine in 2.4, 
and is fine once getty gets ahold of it, it's just during the initial 
bootup phases where it's being used as the console either by the rc scripts 
or by the kernel that seems to go wonky.  It goes out during the initial 
printk output, or sometimes later...exactly when seems to be a bit of a 
random thing.  Also it either causes, or is inputting NULL's or some other 
(consistent) garbage (CRLF? instead of CR?) on these same blades.  So you 
type root<ENTER> and get something like root<NULL><ENTER> or 
root<ENTER><NULL>  For those interested the interaction is with RLX (now 
bought by HP and no longer manufactured new as of a couple months ago) HPC 
Blade Chassis.  I'm guessing that maybe it's fixed somewhere in between 
2.6.8 and 15 (hoping it is) but until I find and backport fixes to 
something with devfs I won't be able to find out.  Part of the problem here 
is yes I'm running and using distros, and in house things, that are built 
on and using devfs.  A big problem I'm having is (yes an issue for Debian 
maintainers but caused by what by all rights looks/should be maintenance 
release of the kernel) the fact that Deb. Sarge's mkinitrd fails because it 
can't find devfs.  That might be because the current system is using devfs 
and mkinitrd just wants to replicate whats loaded/used now, I have to dig a 
lot deeper into that, yes I agree it's a debian-with-2.6.15.1 bug but the 
lack of bugfixes into/with 2.6.8 (or anywhere apparently now :/ ....).


I think I have more kernel bugs and can go on, but I'll just be told 
'upgrade to 2.6.15' which is not an option in many cases if these are 
indeed development releases, if only 'politically', but there are often 
real costs involved.  And with nowhere to put patches that end up in 
maintenance releases we're forced to maintain our own private forks, and 
likely, because of the GPL, also publish these forks and incur all the 
costs associated with that directly, and hope they don't become 
popular/wanted outside of the customer base they're intended for, or skirt 
the GPL, and only allow customers access to this stuff.

> What version are you finding things that work, and then later, not work?
>
> Are you properly looking at the required version of things in the README
> file?
>
> Again, specifics please, otherwise nothing can have a chance of getting
> better.

The bugs are only the small part of what I see as a larger issue though.  I 
know I can get the bugs fixed, atleast in the latest development releases, 
whatever their version numbers are.  I'm in an odd position of working for 
a web hosting company, *and* doing my own Linux consulting as well, and 
maintaining some 'embedded distros' used in these specific niche 
applications.

But pass that along to the customer/client I hear you say.  At that point 
it quickly starts to become cheaper to use VxWorks or something else and 
abandon Linux altogether.  I don't want to do that because ultimately it's 
the strength of the community that makes Linux the better choice.  There's 
also obviously significant time invested in existing work.  I think that 
also Linux is ultimately a much more well featured, and generally stable 
(operating wise as far as I know we can compile and run it on X platform 
and it runs for 6 months without problems) kernel.

I'm not trying to attack any of the developers here at all.


> thanks,
>
> greg k-h
>



--
"Genius might be described as a supreme capacity for getting its possessors
into trouble of all kinds."
-- Samuel Butler

Re: Development tree, PLEASE?

[Christopher Friesen]

Michael Loftis wrote:

> And with nowhere to put patches that end up in 
> maintenance releases we're forced to maintain our own private forks, and 
> likely, because of the GPL, also publish these forks and incur all the 
> costs associated with that directly, and hope they don't become 
> popular/wanted outside of the customer base they're intended for, or 
> skirt the GPL, and only allow customers access to this stuff.

You do realize that if you ship the code along with the product then 
you're fully compliant to the GPL? Your customers can do whatever they 
want with the code, but your obligations are satisfied.

Chris

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 20, 2006 8:00:52 PM +0000 Russell King 
<rmk+lkml@arm.linux.org.uk> wrote:

> On Fri, Jan 20, 2006 at 12:21:40PM -0700, Michael Loftis wrote:
>> I think that it's fine to push the maintenance effort away from the
>> mainline developers, probably even desireable, but then the
>> bugfixing/etc  tends to happen in a disparate manner, off on lots of
>> forks at different  places without them making their way back to some
>> central place.
>
> The responsibility for ensuring that those bugfixes get back to "some
> central place" is with the folk who created the bugfixes.
>
> I've seen this _far_ too many times - it's what I call "the CVS disease"
> and it happens _a_ _lot_ in certain areas.
>
> Developer X uses a CVS tree for his work and has write access to that
> tree.  He finds a bug, and fixes it.  Maybe he writes a good explaination
> of the bug and puts it in the CVS commit comments.  He commits the fix.
> When he's done with development, he walks away and the fix never gets
> submitted.  (Not everyone operates this way, but I know some do.)
>
>
> As a mainline guy myself, I'll say we're all welcome to whatever bug
> fixes come our way.  We just need someone to send them to us with an
> adequate explaination of the bug.

OK, my question though related to that is, where would they be included? 
For most of my cases, latest development doesn't really help.  I'd still 
have to maintain a completely forked kernel.  Would they be included or 
eligible for the 4th digit releases you're talking about?  But then that 
seems that efforts might get really spread out across the many 3rd digit 
releases, making that situation just as bad, or worse, as the previous 
odd/even issues.

>
>> And that seems where we're going with this conversation.  A fork/forks
>> at  various versions to maintain bugfixes and support updates that's
>> (more?)  open to submitters writing patches.  Maintained by a separate
>> group or  party, but with the 'blessing' from mainline to do so.  A
>> place for those  sorts of efforts to be focused, without necessarily
>> involving the primary  developers.
>
> Do you know about the bugfix-only kernel series, which have 4-digit
> versions - 2.6.x.y - which is the compromise to satisfy folk with the
> issue you're bringing up in this paragraph?

I don't see any maintenance releases on 2.6.8 edxcept one which addresses a 
separate issue I ran into I don't see an updated e1000 in 2.6.8 (I'm not 
sure where exactly this particlar e1000 gets supported Manuf ID is intel, 
0x0108 is the device id...IIRC) but I'm pretty sure it's supported in 
2.6.15, can't go much earlier than that because 2.6.9 and later seem to 
have bugs with aic7xxx up until 2.6.14 or 15 (not clear here, I haven't 
done enough testing, so basing that on input from others) as mentioned 
though 2.6.8 also has buglets.  I think 2.6.8.1 actually has a fix for the 
NFS problem I forgot about in my slightly earlier reply to a separate part 
of this thread, though I don't know if Debian has included that in their 
2.6.8 kernels or not.  Not even totally sure that's the issue I was seeing, 
I'm still investigating that too.

I think the four digit bugfix only stuff is an excellent step, and 
necessary.  But the thing that I need more is stable APIs (both userland 
and kernel, and at the kernel<->userland interface) *with* bugfixes and 
(hopefully with) trivial hardware support update backports, like the 
replacement e1000 driver.  And I guess I shouldn't say 'I' need, but 
colleagues need.  And it's not just one company or one project or one 
client/customer.  And not all the issues are the same, but they come back 
to needing somewhere that's kept 'dusted off' but not rearranged (too?) 
regularly.

Re: Development tree, PLEASE?

[Doug McNaught]

Michael Loftis <mloftis@wgops.com> writes:

> I think the four digit bugfix only stuff is an excellent step, and
> necessary.  But the thing that I need more is stable APIs (both
> userland and kernel, and at the kernel<->userland interface) *with*
> bugfixes and (hopefully with) trivial hardware support update
> backports, like the replacement e1000 driver.  And I guess I shouldn't
> say 'I' need, but colleagues need.  And it's not just one company or
> one project or one client/customer.  And not all the issues are the
> same, but they come back to needing somewhere that's kept 'dusted off'
> but not rearranged (too?) regularly.

The point is that this is hard work, and not very interesting.
Commercial distro vendors pay people to do it.  If you want a similar
community effort, but you're not prepared to invest time, money, or
leadership, well, too bad.

And your desire for such a project to be "blessed" makes no sense.
Create your fork, maintain it, and see who else wants to use it.  If
it gets enough users and stays useful, I'm sure that it can be hosted
on kernel.org -- that's really the only kind of "blessing" that there
is.

Remember that the people who maintained 2.2 and 2.4 as "stable"
kernels volunteered to do it and put a *lot* of time into it.  It
didn't just magically happen.

-Doug

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 20, 2006 9:20:19 PM +0100 Jesper Juhl <jesper.juhl@gmail.com> 
wrote:

> On 1/20/06, Michael Loftis <mloftis@wgops.com> wrote:
>>
> [snip]
>> I'm trying to think of a way to relate this better but I just can't.
>> What's needed is a 'target' for incremental updates, things like minor
>> changes, bugfixes, etc.  I feel like supporting entirely new hardware
>
> That's called a vendor kernel.
> You pay the vendor money, the vendor maintains a stable (as in feature
> frozen) kernel, backports bugfixes for you etc.
> Take a look at the RedHat and SuSE enterprise kernels, they seem to be
> what you want.

RedHat's kernel is, I'm sorry, a car wreck of too many patches.  We tried 
that in the hosting environment, had many many gremlins as a result.  Most 
of which are still unresolved.  I've got httpd's and mysqld's that the 
root/listener process uses up almost all of the CPU.  And they're not doing 
anything.

Even without that I'm all for cleaner kernels, hopefully with pretty well 
documented reasons behind changes or clear reasons.  RH is trying to be 
everything, which is fine for them and their intended audience.  I've never 
really been happy with their kernels, nor with their base os.  Many are 
though.

Why can't a community do this though?  I guess the answer is there's no 
reason a community cant, jsut the mainline developers are not going to, 
because it's too much work.

> In my book 'stable' means 'doesn't crash' and 'doesn't break userspace
> without long advance notice', it doesn't mean 'does not evolve/goes
> stale'.
> And in my oppinion the current 2.6 tree succeeds in being a stable kernel.

I think stable should also include bugfixes and updates without having to 
take (potentially, if not certainly) incompatible changes along with that. 
Which yes, is closer to many distro's models.

>
> Besides, I don't agree with your view that we break userspace all the
> time as you seem to be saying in several of your mails, quite the
> opposite - a lot of work goes into *not* breaking userspace.
> Just take a look at how syscalls are maintained, even old obsolete
> ones stay in place since removing them would break userspace. Stuff in
> /proc does not get changed since that would potentially break
> userspace. Look at the fact that you can still run ancient a.out
> binaries on a recent 2.6.x kernel.
> Even internal kernel APIs usually stay around with __deprecated or as
> wrappers around new APIs for extensive periods of time.
> And when stuff is removed it's announced for ages in advance. That
> devfs would be removed was announced several *years* before it
> actually happened. That old OSS drivers will be removed (but only for
> hardware that has ALSA equivalents) has been announced months ago and
> the removal won't happen for several months (at least) yet.
>
> I'd say the kernel tries damn hard at maintaining backwards
> compatibility for userspace.
> It tries less hard, but still makes a pretty good effort, for internal
> APIs, but the real solution to the internal API churn is to get your
> code merged as it'll then get updated automagically whenever something
> changes - people who remove or change internals try very hard to also
> update all (in-tree) users of the old stuff - take a look at when I
> removed a small thing like verify_area() if you want an example.

The only argument I have is one that won't fly at all here on LKML and 
that's about all the corporate sponsors the LK has that are also doing 
custom closed source modules that are only useful for their particular 
hardware.  I'm working with more than one such company now, if they want to 
step forward and name themselves they can, but I can't name them.  Without 
these companies paying various salaries and developing using Linux and 
pushing bugfixes back/etc on the open source portions of their products it 
would be a different landscape.

>> It's horrificly expensive to maintain large numbers of machines (even if
>> it's automated) as it is.  If you're doing embedded development too or
>> instead, it gets even harder when you need certain bugfixes or minor
>> changes, but end up having to redevelop things or start maintaining your
>> own kernel fork.
>>
> The solution here is to get your code merged in mainline.

Most of it can't.  Or simply won't be accepted.  Noone else has use for a 
PPC where the GPIO is driving a custom data acquisition FPGA, or things of 
that nature.  Some of it is the same old problem of proprietary firmware 
and IP.  Some of it isn't.  Most of it is just simply useless to everyone 
but the device's manufacturer, and thus wouldn't be maintained anyway, much 
less accepted.  I guess for those cases that it *might* be accepted and can 
be exported I'll have to decide where the tradeoff occurs between answering 
external questions about hardware that doesn't exist outside of these 
devices and apps.

There again, this is still just one part of the problem as a whole 
discussed in this thread.

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 20, 2006 9:20:19 PM +0100 Jesper Juhl <jesper.juhl@gmail.com> 
wrote:

<..>
> The solution here is to get your code merged in mainline.

The thing I see is the whole problem of not having a central stable 
bugfixed line of code, or lines of code where the breaks in the APIs are 
pretty clearly defined.

>
> [snip]
>
> --
> Jesper Juhl <jesper.juhl@gmail.com>
> Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
> Plain text mails only, please      http://www.expita.com/nomime.html
>



--
"Genius might be described as a supreme capacity for getting its possessors
into trouble of all kinds."
-- Samuel Butler

Re: Development tree, PLEASE?

[Dmitry Torokhov]

On 1/20/06, Michael Loftis <mloftis@wgops.com> wrote:
>
>
> --On January 20, 2006 9:20:19 PM +0100 Jesper Juhl <jesper.juhl@gmail.com>
> wrote:
>
> > On 1/20/06, Michael Loftis <mloftis@wgops.com> wrote:
> >>
> > [snip]
> >> I'm trying to think of a way to relate this better but I just can't.
> >> What's needed is a 'target' for incremental updates, things like minor
> >> changes, bugfixes, etc.  I feel like supporting entirely new hardware
> >
> > That's called a vendor kernel.
> > You pay the vendor money, the vendor maintains a stable (as in feature
> > frozen) kernel, backports bugfixes for you etc.
> > Take a look at the RedHat and SuSE enterprise kernels, they seem to be
> > what you want.
>
...
> RH is trying to be everything, which is fine for them and their intended audience.  I've never
> really been happy with their kernels, nor with their base os.  Many are
> though.
>
> Why can't a community do this though?  I guess the answer is there's no
> reason a community cant, jsut the mainline developers are not going to,
> because it's too much work.
>
...
>
> I think stable should also include bugfixes and updates without having to
> take (potentially, if not certainly) incompatible changes along with that.

Are you volunteering?

--
Dmitry

Re: Development tree, PLEASE?

[Vincent Hanquez]

On Fri, Jan 20, 2006 at 02:27:50PM -0500, Ben Collins wrote:
> So put me in for +1 on the current development model.

I'ld like to also +1 the current development model.

However I think the 2.6.X should become 2.X; the 6 is starting to be
meaningless, and it's not going to be upgraded ever with the current
development model.

this leave 2.X.Y "namespace" to the current stable team (same
development model as the 2.6.X.Y).

-- 
Vincent Hanquez

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 20, 2006 8:25:34 PM +0000 Russell King 
<rmk+lkml@arm.linux.org.uk> wrote:

> On Fri, Jan 20, 2006 at 10:14:15AM -0700, Michael Loftis wrote:
>> It's easier for an embedded system especially to pick a target, and then
>> stay with it.  Later when a new major version comes out the time can
>> then  be invested ONCE to redevelop what needs redeveloping, which is
>> easier to  do (yes I'm speaking from a business standpoint, sorry, but
>> someone has to)  and to sell to management than nickel-and-dime to death
>> of trying to follow  a development tree.
>
> Please note that the current development strategy suits embedded folk.
>
> With the old model, embedded folk could not wait two years for a (eg)
> 2.5 kernel to become a 2.6 kernel and then "stabilise".  In two years,
> the new SoC is already in full-use in multiple applications and folk
> have been and long gone developing their products.
>
> So what happens is a massive conflict of interest - do we put
> $mass-of-new-features into 2.4 kernels at the risk of destabilising
> the current users, or do we push it into 2.5 and wait two or more
> years before folk start using that code.
>
> Like it or not, the latter causes a _lot_ of difficulties for a _lot_
> of people, so much so that it's an economic disaster unless you're a
> large corporation.  The former is also a non-starter because then you
> end up with folk complaining that it should be in the development
> branch.  It's a no-win situation - you can't satisfy everyone.

Yeah this is true.

> So, our current model is a compromise - you get _told_ that things
> will change well in advance, and if you choose to ignore those warnings
> (or don't respond to them) that's entirely your own problem.

I guess part of the problem is this process is still relatively new and all 
the pointers I'm used to no longer apparently apply.

>
> It's like a stick and carrot scenario - see
> http://everything2.com/index.pl?node=stick%20and%20carrot
>
> The carrot in this case is the notice about devfs removal.  The
> stick is the actual devfs removal.  One's pleasant, the other isn't.
> Which you take notice of is entirely your choice.

In atleast one case of my scenarios it's not heh.  I've no choice but to 
take 2.6.15(.x) but I have to build custom debian installers and maybe 
still bring in newer e1000 sources from intel (still awaiting that build to 
finish to find the results of that) but there's issues with devfs there 
too, the only machine capable of testing needs devfs to boot and mkinitrd 
fails miserably at the moment, and I'm not sure why, which I am 
investigating more fully as well.

Like I said the problem from my point of view is needing a few trivial 
changes, but having to take a whole new ball of wax, and all the testing 
and verification work that takes for the hosting environments (maintaining 
multiple kernels internally was long ago given up because it's impossible 
to keep track of easily) here because we have to go completely away from 
the vendor kernels and for the places where I'm maintaining embedded 
(admittedly mostly x86 devices are my primary responsibility, though lately 
I've been signed into a few ARM projects) though is the most painful since 
there is a need to track the current kernel with bugfixes, and there's no 
vendor to act as a buffer, and no central place to find these 
updated/patched kernels which also measns if we make changes that cause us 
to become a fork we might be legally bound to atleast provide a patch for 
the core portions under GPL.

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 20, 2006 4:40:50 PM -0500 Doug McNaught <doug@mcnaught.org> 
wrote:

> Michael Loftis <mloftis@wgops.com> writes:
>
>> I think the four digit bugfix only stuff is an excellent step, and
>> necessary.  But the thing that I need more is stable APIs (both
>> userland and kernel, and at the kernel<->userland interface) *with*
>> bugfixes and (hopefully with) trivial hardware support update
>> backports, like the replacement e1000 driver.  And I guess I shouldn't
>> say 'I' need, but colleagues need.  And it's not just one company or
>> one project or one client/customer.  And not all the issues are the
>> same, but they come back to needing somewhere that's kept 'dusted off'
>> but not rearranged (too?) regularly.
>
> The point is that this is hard work, and not very interesting.
> Commercial distro vendors pay people to do it.  If you want a similar
> community effort, but you're not prepared to invest time, money, or
> leadership, well, too bad.

It sounds like that's what it's coming down to.  I'm willing to I just as 
anyone, need to be careful not to bite off too much.  And right now this 
sounds like it might be.

> And your desire for such a project to be "blessed" makes no sense.
> Create your fork, maintain it, and see who else wants to use it.  If
> it gets enough users and stays useful, I'm sure that it can be hosted
> on kernel.org -- that's really the only kind of "blessing" that there
> is.
>
> Remember that the people who maintained 2.2 and 2.4 as "stable"
> kernels volunteered to do it and put a *lot* of time into it.  It
> didn't just magically happen.

I know, and I'm incredibly grateful for that.  Heck up until just a year 
ago there was a 2.2.x box in the corner at home.  Heroic effort on those 
persons parts.

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 20, 2006 5:00:54 PM -0500 Dmitry Torokhov 
<dmitry.torokhov@gmail.com> wrote:
> Are you volunteering?

I'd like to, but it's more than just an issue of time.  The issue of how in 
the heck to host such a thing is going back and forth now.  This may yet 
happen.  not...no not quite what I'd planned when I first started this 
thread, but not completely outside of what I was expecting.

You want something done (right...or at all ;) ) ya gotta do it yourself.  I 
just don't want to completely be the lone ranger.  If I do this as a public 
effort it has to be useful to others.  Because if I'm doing it as a private 
effort, I can get sloppy and I don't have to maintain a web page or have 
hosting bandwidth for it or repositories outside of the normal private svn 
repo I keep. :)


Thanks for all those who've participated (and continue to participate) in 
this discussion/debate.  It's been very informative even if I started it 
out badly.

Re: Development tree, PLEASE?

[Horst von Brand]

Michael Loftis <mloftis@wgops.com> wrote:

[...]

> Lots of things still out there depend on devfs.

Nothing in the regular kernel, nothing in any sane distribution.

devfs has been slated for the axe for a /very/ long time.

>                                                  So now if I want to
> develop my kmod on recent kernels I have to be in the business of
> maintaining a lot more userland stuff, like mkinitrd, installers,
> etc. that have come to rely on devfs.

Your own fault that you didn't heed the warnings about stuff to be deleted
and didn't keep up with the regular development around the kernel. Not the
kernel hackers' responsibility.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: Development tree, PLEASE?

[Horst von Brand]

Michael Loftis <mloftis@wgops.com> wrote:
> --On January 20, 2006 11:43:31 AM -0800 Greg KH <greg@kroah.com> wrote:
> > On Fri, Jan 20, 2006 at 10:14:15AM -0700, Michael Loftis wrote:
> >> The problem here is I'm spending a lot of my time lately fixing things
> >> that  shouldn't need fixing.  Things that are/were developed against
> >> what was  supposed to be a stable major version and has been turned into
> >> a  development version.

> > What specifically are you "fixing"?

> At this point I'm looking at bugs in the aic7xxx driver, it mostly
> works in 2.6.8, occasionally locking up my tape subsystem, it's
> apparently fixed in 2.6.15 or 2.6.15.1, I need to look closer into
> that, and backport it because of the devfs issue I don't think I can
> take 2.6.15/2.6.15.1 whole hog.  A decent amount of ARM stuff moving
> around between even just 2.6.11 and 2.6.13 (admittedly that's a gripe
> for ARM) making development for that port very painful (there's talk
> of switching to something else because of all of this for those
> projects) -- no specifics on the ARM stuff as I'm not the developer
> directly involved with most that, I'm just doing some PHY code, which
> will eventually be submitted back to the mainstream ARM (still in
> product development) and he's indisposed today/at the moment, I'll try
> to get him to summarize those issues so I can relay them to the list.

You do realize that doing all that is much more work than just fixing
whatever you are doing wrong with the configuration of your machines, do
you? And that that job /will/ get increasingly harder as time goes by?

Better install anew, it looks like the current situation is beyond hope.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: Development tree, PLEASE?

[Russell King]

On Fri, Jan 20, 2006 at 01:56:12PM -0700, Michael Loftis wrote:
> A decent amount of ARM stuff moving around between even just 2.6.11 
> and 2.6.13 (admittedly that's a gripe for ARM) making development for that 
> port very painful

What you're complaining about here seems to be:

1. the cleanup of the entry and debug code - TRIVIAL - it's a
   cut-n-paste job.  No interface change.  Estimated time to
   resolve: 5 minutes
2. moving the machine specific boot makefile parameters into
   arch/arm/mach-* - TRIVIAL - it's a cut-n-paste job.  No interface
   change.  Estimated time to resolve: 5 minutes.
3. removing messy macros for the machine description.  Slightly less
   trivial because you need to do some investigation and then an
   exercise of about 10 subsitutions in one file.  Estimated time
   to resolve: 30 minutes.

All changes listed above to lower the long term maintainence burden of
the kernel _and_ make it easier to port to new SoCs.

Ok, so, let's be generous - call it one hour.  Are you _really_ griping
about one hour's work on your part being "very painful"?

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Re: Development tree, PLEASE?

[Bernd Petrovitsch]

On Fri, 2006-01-20 at 12:10 -0700, Michael Loftis wrote:
> --On January 20, 2006 2:03:52 PM -0500 Valdis.Kletnieks@vt.edu wrote:
> > But you're perfectly happy to make the kernel developers do the
> > equivalent thing when they have to maintain 2 forks (a stable and devel).
> > Go back and look at the status of the 2.5 tree - there were *large*
> > chunks of time when 2.4 or 2.5 would get an important bugfix, but the
> > other tree wouldn't get it for *weeks* because of the hassle of
> > cross-porting the patch.
> 
> Weeks is better than never, and still better than commercial vendors. ;)

Especially if someone else is doing the work (- to get back to the
point).

	Bernd
-- 
Firmix Software GmbH                   http://www.firmix.at/
mobil: +43 664 4416156                 fax: +43 1 7890849-55
          Embedded Linux Development and Services

Re: Development tree, PLEASE?

[Greg KH]

On Fri, Jan 20, 2006 at 01:56:12PM -0700, Michael Loftis wrote:
> 
> 
> --On January 20, 2006 11:43:31 AM -0800 Greg KH <greg@kroah.com> wrote:
> 
> >On Fri, Jan 20, 2006 at 10:14:15AM -0700, Michael Loftis wrote:
> >>The problem here is I'm spending a lot of my time lately fixing things
> >>that  shouldn't need fixing.  Things that are/were developed against
> >>what was  supposed to be a stable major version and has been turned into
> >>a  development version.
> >
> >What specifically are you "fixing"?
> 
> At this point I'm looking at bugs in the aic7xxx driver, it mostly works in 
> 2.6.8, occasionally locking up my tape subsystem, it's apparently fixed in 
> 2.6.15 or 2.6.15.1, I need to look closer into that, and backport it 
> because of the devfs issue I don't think I can take 2.6.15/2.6.15.1 whole 
> hog.

devfs is long dead and gone.  It's going to be much easier for you to
probably just change your userspace config to handle this.

If you need any help doing this, please ask on the linux-hotplug-devel
mailing list, where lots of people can help you out.

Or, just add the CONFIG_DEVFS config option to your .config, and build
devfs into the 2.6.15 release.  The code is still there...

> As far as fixing there are modules that have been developped to run various 
> embedded peripherals that must be reworked to use the newer kernel 
> versions, which wouldn't be a problem if there weren't various other fixes 
> that were needed which means moving up point revs.  Most of these other 
> bugs are external to my work, but they affect my work.  The modules are 
> completely isolated from the rest of the kernel though and they're for very 
> particular hardware for different clients.

There's nothing we can do about out-of-the-tree kernel versions, see
Documentation/stable_api_nonsense.txt about why you should get those
modules into the main kernel tree.

And before you say, "but they are only for some very odd and not popular
devices, no one would want them in the kernel tree!", remember that we
have whole arches that are only run by about 2 users.  I know
specifically about a few drivers that only work on 1 device in the whole
world.  So this isn't a good excuse :)

Your other issues sound like they will be fixed with the latest kernel
version, if not, please let us know.

> I think I have more kernel bugs and can go on, but I'll just be told 
> 'upgrade to 2.6.15' which is not an option in many cases if these are 
> indeed development releases, if only 'politically', but there are often 
> real costs involved.

Then what do you expect us to do?  And what are those costs?

> And with nowhere to put patches that end up in maintenance releases
> we're forced to maintain our own private forks, and likely, because of
> the GPL, also publish these forks and incur all the costs associated
> with that directly, and hope they don't become popular/wanted outside
> of the customer base they're intended for, or skirt the GPL, and only
> allow customers access to this stuff.

I hear sf.net hosts patches for free :)  I know of a specific big distro
vendor that likes to burry their patches there to apease the letter of
the GPL, and make it hard for others to figure out where the code is...

thanks,

greg k-h

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 20, 2006 11:17:57 PM +0000 Russell King 
<rmk+lkml@arm.linux.org.uk> wrote:

> On Fri, Jan 20, 2006 at 01:56:12PM -0700, Michael Loftis wrote:
>> A decent amount of ARM stuff moving around between even just 2.6.11
>> and 2.6.13 (admittedly that's a gripe for ARM) making development for
>> that  port very painful
>
> What you're complaining about here seems to be:
>
> 1. the cleanup of the entry and debug code - TRIVIAL - it's a
>    cut-n-paste job.  No interface change.  Estimated time to
>    resolve: 5 minutes
> 2. moving the machine specific boot makefile parameters into
>    arch/arm/mach-* - TRIVIAL - it's a cut-n-paste job.  No interface
>    change.  Estimated time to resolve: 5 minutes.
> 3. removing messy macros for the machine description.  Slightly less
>    trivial because you need to do some investigation and then an
>    exercise of about 10 subsitutions in one file.  Estimated time
>    to resolve: 30 minutes.
>
> All changes listed above to lower the long term maintainence burden of
> the kernel _and_ make it easier to port to new SoCs.
>
> Ok, so, let's be generous - call it one hour.  Are you _really_ griping
> about one hour's work on your part being "very painful"?

As long as it isn't wash rinse repeat, but in development kernels it tends 
to be.  That's the pain point.  It's not one single huge problem, it's the 
constant stream of little ones that we try to avoid.

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 20, 2006 3:27:03 PM -0800 Greg KH <greg@kroah.com> wrote:

> devfs is long dead and gone.  It's going to be much easier for you to
> probably just change your userspace config to handle this.
>
> If you need any help doing this, please ask on the linux-hotplug-devel
> mailing list, where lots of people can help you out.
>
> Or, just add the CONFIG_DEVFS config option to your .config, and build
> devfs into the 2.6.15 release.  The code is still there...

That's going to be more or less what will happen for the cases I need to go 
to 2.6.15 and can't just apply a short patch or cp of new source.  I have 
to patch up Kconfig's though too since make-kpkg will run a make oldconfig 
which will remove options it doesn't see, normally a good thing.  For other 
problems it looks like I'll begin to maintain my own kernel forks atleast 
in the short run.

> There's nothing we can do about out-of-the-tree kernel versions, see
> Documentation/stable_api_nonsense.txt about why you should get those
> modules into the main kernel tree.

I know, and I don't expect the LK people to maintain any of my code in any 
way, nor any member outside of the teams it was developed in and for.

> And before you say, "but they are only for some very odd and not popular
> devices, no one would want them in the kernel tree!", remember that we
> have whole arches that are only run by about 2 users.  I know
> specifically about a few drivers that only work on 1 device in the whole
> world.  So this isn't a good excuse :)

OK well, this I hadn't realised, my impression was that the case was mostly 
or entirely opposite of this.  That a new bit had to have really good buy 
in before it could get anywhere near any mainline development, much less 
release cycles.  I'll have to get really snuggly with the whole release 
policy again, I was under the (now I see very wrong, I'm sorry gents) 
impression there wasn't this major of a shift going on.  I simply don't 
have the bandwidth to follow l-k most of the time.

> Your other issues sound like they will be fixed with the latest kernel
> version, if not, please let us know.

Except for any new ones I might find ;)  Which is where this whole thread 
got started in my mind anyway.

>
>> I think I have more kernel bugs and can go on, but I'll just be told
>> 'upgrade to 2.6.15' which is not an option in many cases if these are
>> indeed development releases, if only 'politically', but there are often
>> real costs involved.
>
> Then what do you expect us to do?  And what are those costs?

Any idea how much work it takes to re load test a kernel, do all the 
required research to make sure you're not introducing new bugs, etc? 
Usually, per arch, for me, it takes about a week, sometimes two.  This last 
round I rushed myself and decided not to test the NFS portion and got bit 
by that see some related NFS posts on that that I made if you're interested 
in that saga, sounds like 2.6.8.1 might fix this...I need to figure out 
what debian policy is in getting these new fourth dot changes applied and 
mainstreamed but that's a separate issue altogether.

Yes part of it is internal distribution policy, but it's also just plain 
good sense.  When all the new development and radical changes are happening 
in the same place you have to look for bug fixes it actually becomes more 
expensive to deploy because you ultimately end up with a lot more change 
between kernel upgrades.

It sounds like I'll atleast be trying to provide such a 'stable patched 
kernel' tree fork area for people to point at, and see if there's enough 
interest to keep it up.  I'm not certain about being able to publicly 
maintain such a beast myself.

> I hear sf.net hosts patches for free :)  I know of a specific big distro
> vendor that likes to burry their patches there to apease the letter of
> the GPL, and make it hard for others to figure out where the code is...

Now now, leave the skeletons alone. ;)  They seem perfectly happy where 
they're at.  Not sure if SF's system has improved any, but it used to be 
pretty clunky to attempt to publish and maintain patch sets and releases, 
or it felt like it to me.  Then again, I have a severe aversion to web UIs 
in that regard.

Thanks for the assistance guys.  Atleast I've got a direction instead of 
being backed completely into a wall and have a lot deeper understanding of 
some of the feelings of the group on all of this.

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 21, 2006 12:20:51 AM +0100 Bernd Petrovitsch <bernd@firmix.at> 
wrote:

> Especially if someone else is doing the work (- to get back to the
> point).

Yup, none of us has time to single-handedly manage such a large 
security/bugkeeping effort.  I just want to see it done in the community 
since I'm a user, and developer using, three different distro's for now and 
the forseeable future, and potentially a fourth.  Not with 100% overlap 
but...certain distros lend themselves better to certain things.

Re: Development tree, PLEASE?

[Russell King]

On Fri, Jan 20, 2006 at 04:33:38PM -0700, Michael Loftis wrote:
> As long as it isn't wash rinse repeat, but in development kernels it tends 
> to be.  That's the pain point.  It's not one single huge problem, it's the 
> constant stream of little ones that we try to avoid.

So what you're basically saying is that we should make zero changes
to the kernel, because any change (even a minor bug fix) may cause
you to need to do some work.  Should we just increment the version
number every 3 months then?

Maybe we could do this _if_ folk would stop working on the kernel,
wanting it to run on their latest creations.

The fact is that in the ARM world, everyone wants a stable kernel
which has support for all the features in the SoC de jour that
they're using.  That previous sentence is self-contradictory - it's
an impossible scenario.  You can't have a kernel which supports the
latest features without progressive and continuous change.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Re: Development tree, PLEASE?

[Russell King]

On Fri, Jan 20, 2006 at 04:52:00PM -0700, Michael Loftis wrote:
> --On January 20, 2006 3:27:03 PM -0800 Greg KH <greg@kroah.com> wrote:
> >And before you say, "but they are only for some very odd and not popular
> >devices, no one would want them in the kernel tree!", remember that we
> >have whole arches that are only run by about 2 users.  I know
> >specifically about a few drivers that only work on 1 device in the whole
> >world.  So this isn't a good excuse :)
> 
> OK well, this I hadn't realised, my impression was that the case was mostly 
> or entirely opposite of this.  That a new bit had to have really good buy 
> in before it could get anywhere near any mainline development, much less 
> release cycles.  I'll have to get really snuggly with the whole release 
> policy again, I was under the (now I see very wrong, I'm sorry gents) 
> impression there wasn't this major of a shift going on.  I simply don't 
> have the bandwidth to follow l-k most of the time.

In the case of ARM machine types, it is the opposite of Greg's
statement.  There's getting on for 1000 ARM machine types.  We have
some 72 machine types currently merged and buildable.

If everyone merged their little-used ARM machine type - say we got
to 250 types, the maintainence burden on _me_ personally would be,
to put it mildly, prohibitive.  Rather than fixing up all machine
support code when I made any change, I'd ignore most of them and
just do the ones I was interested in.

That results in most merged code falling into a state where it's
essentially broken, and a _lot_ more folk whinging about change.

Alternatively, as I said in my other recent mail, we could stagnate.

Here's a thought - if this is soo important, would you pay me to
stagnate the ARM part of the kernel tree? 8)  (rmk thinks... could
run this as a "highest bidder gets their way") 8)

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 20, 2006 11:55:20 PM +0000 Russell King 
<rmk+lkml@arm.linux.org.uk> wrote:

> On Fri, Jan 20, 2006 at 04:33:38PM -0700, Michael Loftis wrote:
>> As long as it isn't wash rinse repeat, but in development kernels it
>> tends  to be.  That's the pain point.  It's not one single huge problem,
>> it's the  constant stream of little ones that we try to avoid.
>
> So what you're basically saying is that we should make zero changes
> to the kernel, because any change (even a minor bug fix) may cause
> you to need to do some work.  Should we just increment the version
> number every 3 months then?
>
> Maybe we could do this _if_ folk would stop working on the kernel,
> wanting it to run on their latest creations.
>
> The fact is that in the ARM world, everyone wants a stable kernel
> which has support for all the features in the SoC de jour that
> they're using.  That previous sentence is self-contradictory - it's
> an impossible scenario.  You can't have a kernel which supports the
> latest features without progressive and continuous change.
>

That's not quite what I want/need.  I want a stable kernel in all respects, 
one that has bugfixes and minor changes, for every day use, for building or 
deploying embedded systems on.  Yes for people who've got the SoC du juour 
the model is better, but for us poor buggers who couldn't give a crud about 
the latest greatest SoC it's a pain.

I think we can just let this thread die because apparently I'm in the very 
unvocal minority that is hurt and this change is costing more than others.

Right now as it sits, you have to bleed, or you don't get anything.  I 
think some more middle ground can be found...I'm just not totally sure it's 
wanted now after this line of discussion.

Re: Development tree, PLEASE?

[Lars Marowsky-Bree]

On 2006-01-20T17:05:02, Michael Loftis <mloftis@wgops.com> wrote:

> Right now as it sits, you have to bleed, or you don't get anything.  I 
> think some more middle ground can be found...I'm just not totally sure it's 
> wanted now after this line of discussion.

The usual suggestion: feel free to pick a kernel of your choice, and
keep maintaining it and backporting fixes to it, and make your tree
available for others.

You see, _nobody_ stops you (or someone else, for that matter) from
doing so. And if the need really exists, the users will come, and the
help with maintaining it will appear.

So far, people keep complaining that noone does it for them. Right. That
won't happen. If you want it, step up.


Sincerely,
    Lars Marowsky-Brée

-- 
High Availability & Clustering
SUSE Labs, Research and Development
SUSE LINUX Products GmbH - A Novell Business	 -- Charles Darwin
"Ignorance more frequently begets confidence than does knowledge"

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 20, 2006 6:08:52 PM +0100 Gábor Lénárt <lgb@lgb.hu> wrote:

> Though I'm not a kernel developer let me allow to comment this based on
> my experiences as well.
>
> On Fri, Jan 20, 2006 at 08:17:40AM -0700, Michael Loftis wrote:
>> OK, I don't know abotu others, but I'm starting to get sick of this
>> unstable stable kernel.  Either change the statements allover that were
>
> What kind of instability have you got? I haven't had any instability since
> at least a year or so, or if there was it was some kind of hardware fault.
> In fact, many machines (like an Armada E500 notebook and some servers as
> well) seems to be stable which was NOT in case of 2.4 kernels! So for our
> experience at our workplace, 2.6 seems to be much more usable than 2.4.x
> kernels (ok, it may be caused by "newer" hardwares, on quite old machines
> I can't show major difference in stability between 2.4 and 2.6)

There's two parts of stable in most of the development world, runs on my 
hardware reliably/runs on most hardware reliably is one part, the other 
part is limited change, usually limited to bugfixes and minor feature fixes 
or updates.  This means that instead of having to take how ever many 
(probably thousands on thousands) of lines of difference, and any of those 
potential new bugs etc, to a much reduced set that just deals with specific 
subsections in order to close specific bugs.  Be it a minor change to fix 
support for a new PCI ID, or a a buffer overflow.  API changes or 
relocation of headers and such would be kept out of a stable branch.  It's 
that second part I hear see and have objections about with 2.6 as it sits. 
There's no 'place' for bugfixes to centralize.  I know that a number of my 
problems are fixed in later kernels, but there's a LOT of fairly large 
change between where I am, and where current is.  Far more than would be in 
a normal stable piece of software.

<...>

> Ah, I see your point. But is it really a BIG problem? I mean please
> mention some *real* issue/story confirm your opinion. Sure, you can find,
> but also compare it with the advantages of new development model, since
> there is nothing in the world which is only have advantages neither
> something which only has disadvantages ... The would is not black or
> white, but a great spectrum of gray shades.

Yup, from 2.6.8 sometime after 2.6.8 aic7xxx is pretty clearly broken from 
many reports I've seen, it was finally fixed in 2.6.15 (I do not know hte 
bug exactly, sorry, I'm using others reports).  In 2.6.8 it's a little 
broken, but mostly working.  If there had been a major bug between there 
and .15 that required me to upgrade to close a security hole I'd have been 
stuck, unable and impossible to upgrade, for that one reason alone.  Worse 
than that because there is so much major change now I have to stress test 
basically every kernel before we can actually start to use it at my day 
job.  We host well over 10k busy mostly dynamic (PHP, Perl, Miva, other 
stuff) web sites on a cluster of Linux based servers.  If there's subtle 
problems they show up in a big way usually, and having them in production 
is not acceptable.

If there was a stable branch with a low change rate then it's easy to track 
the changes even just 'visually' without necessarily having to go through a 
whole stress testing procedure.

I'm not saying an increased/rapid development pace is a bad thing.  I'm 
just wondering where the refuge from that is for systems that don't need, 
don't want, or really can't have that level of change happening, without 
resorting to maintaining ones own kernel fork.  It's one thing to 
compile/package ones own custom packages for a distro when you're already 
using custom kernels or not using their kernel, or even if you're just 
using your own, it's another to actually really maintain your own tree by 
oneself.

Yes I agree with what others have said, it gets to be more and more work. 
Perhaps something along the lines of 3 6 or 9 months with 1 or 2 'community 
supported stable releases'  -- in my day job i'd personally like to see 
longer terms, but ~6 months would be manageable atleast as to major change 
bumps.

>
>> Yes, I'm venting some frustrations here, but I can't be the only one.  I
>> know now I'm going to be called a troll or a naysayer but whatever.  The
>> fact is it needs saying.  I shouldn't have to do major changes to
>> accomodate sysfs in a *STABLE* kernel when going between point revs.
>> This  is just not how it's been done in the past.
>
> sysfs should not used by an average application, I guess, so it's not a
> major point

No not in itself.  I'm looking at a LOT bigger issue.  Everyone seems to 
want to look at and work with a tiny little piece, but there's the bigger 
issue of what is stable.  What does it mean to be stable.  In the minds of 
the people I've worked with directly it's always meant the two things 
outlined earlier.  Runs reliably, and is in a maintenance mode (yes, 
synonymous maybe with stagnant).  That means that it is necessarily a fork, 
away from the main line(s) of development and work.  But it serves a 
purpose in many environments.  It's utility is lost in the average desktop, 
but in the corporate desktop, in servers, embedded devices, commercial 
products, etc. it's a big win.

Re: Development tree, PLEASE?

[Alan Cox]

On Gwe, 2006-01-20 at 13:56 -0700, Michael Loftis wrote:
> and is fine once getty gets ahold of it, it's just during the initial 
> bootup phases where it's being used as the console either by the rc scripts 
> or by the kernel that seems to go wonky.  It goes out during the initial 

A bug where the serial console could get stuck on SMP IFF a kernel and a
non kernel message were output at the same time did get fixed
(yesterday) other than that I'm not aware of any problems in this area
but the maintainer may have more ideas.

Diff is tiny if you want to see if that is what you hit, although it
would be remarkable co-incidence and luck if it was ;)

> printk output, or sometimes later...exactly when seems to be a bit of a 
> random thing.  Also it either causes, or is inputting NULL's or some other 
> (consistent) garbage (CRLF? instead of CR?) on these same blades.  So you 

Never seen CR, nul reported. Would the blades happen to use rlogin to
manage this remote serial do you know ?

> I think I have more kernel bugs and can go on, but I'll just be told 
> 'upgrade to 2.6.15' which is not an option in many cases if these are 
> indeed development releases, if only 'politically', but there are often 
> real costs involved.  And with nowhere to put patches that end up in 

Its hard to maintain an old release and just merge all the fixes into it
backporting when neccessary. At the kernel summit before 2.6 this was
discussed a lot. There are a small number of groups of people who wanted
this for the long term. Said groups either maintain such trees and sell
support/services for money, or rebuild the output of the former as a
community project.

It therefore seemed reasonable that those who want it should bear the
cost, or figure out how to maintain such backports between themselves.

> maintenance releases we're forced to maintain our own private forks, and 
> likely, because of the GPL, also publish these forks and incur all the 
> costs associated with that directly, and hope they don't become 
> popular/wanted outside of the customer base they're intended for, or skirt 
> the GPL, and only allow customers access to this stuff.

The GPL is very careful about this. If you ship the sources to your
customers then you have done your duty. If your customers choose to give
it to others so be it. If the others ask you for changes then I believe
the phrase is "business opportunity". 

> whatever their version numbers are.  I'm in an odd position of working for 
> a web hosting company, *and* doing my own Linux consulting as well, and 
> maintaining some 'embedded distros' used in these specific niche 
> applications.

Embedded can be more problematic because it is harder to spread the
load, but there are communities of people who looked at things like Red
Hat Enterprise Linux and decided that they could use the code but didn't
currently need/want the training, support and services that are what
really makes it. One obvious example is Centos which is a community tree
derived from the RHEL work, rebuilt, rebranded without
support/services/etc and downloadable for free.

Alan

Re: Development tree, PLEASE?

[Matthew Frost]

--- Michael Loftis <mloftis@wgops.com> wrote:

> 
> 
> --On January 20, 2006 4:29:44 PM +0000 James Courtier-Dutton 
> <James@superbug.co.uk> wrote:
> 
> > It is unclear what you are really ranting about here. The "stable"
> > kernel is stable or at least as stable as it is going to be. It is
> > left to distros to make it even more stable. The interface to user
> > land has not changed.
> > If all you are ranting about is the move from devfs to udevd, 
> > then all the user land tools dealing with them have been updated
> > already.
> 

Amen.  devfs has sat in the kernel for an entire major release, 2.4, and
gotten dustier and dustier.  It was deprecated when gregkh asked to
remove it *1 YEAR* before the community let him.  1 whole year prior to
the "last chance" email.  You cannot state that it is anyone's fault but
your own that you missed that.  Your complaint, as far as any of us can
dredge a legitimate complaint out of the noise, is a devfs complaint.  If
you wish to protest the removal of an unused, unmaintained, and
long-deprecated subsystem, title your message such that we understand
that you are complaining about that subsystem.  DO NOT protest about the
stability of the kernel series.  That is not your legitimate issue.  If
you have stability problems, find the cause of the system instability and
email the list about it, to get it fixed.  If you have a complaint about
a new commit in a current series, say 2.6.15, or a patch thereunto
appertaining, note the patch, and discuss why it is problematic.

But if all you're whinging about is devfs being out of 2.6 midstream when
 nobody's using it and there are options for those who do, tell us so. 
Then, we can tell you to used udevd or ndevfs and get out of our hair.

> That's the nail on the head exactly.  Why is this being done in an even
> numbered kernel?  This represents an API change that has knock on well 
> outside of the kernel, and should be done in development releases.  Why
> is it LK is the only major project (that I know of) that does this? 
This
> is akin to apache changing the format of httpd.conf and saying in say
> 1.3.38 and saying 'well we made the userland tools too.'

The kernel is a law unto itself because the kernel is a project like
precious few others.  Name a project like unto the kernel, with the
degrees of similarity and applicability for the given solution, and the
solution as it should logically apply to the kernel for the kernel's
similarity to that project, and maybe we'll talk.  Apache is nothing
like.  Linus has long said that "X does it this way" is not a reason to
do it that way.

> 
> >
> > What is the real specific problem you are having?

(Please, do tell, if it's not devfs)

> 
> Well there's a whole grab bag of them that I'll be getting to over the
> next few months, but the most immediate is the fact that I've gotten
new 
> hardware from a venduh that requires me to build a new Debian installer
> and new debian kernels.  I also have custom packages that depend on
devfs
> being there and now it's not.
> 

No.  Wrong.  If there're a whole grab bag, as you say, then you should
post each, as a separate issue, possibly with consistent proposals for
fixing them.  Follow protocol.  Posting a "The Kernel Is Falling" email
gets people riled up, makes you look foolish, and gets nothing fixed. 
Noise.  Send signal.  We'll wait.

Your "venduh" hardware really is a separate issue, unless you mean to say
that your complaint is that you got too used to devfs and now don't know
how to write drivers.  And it doesn't play nice with Debian?  Why, this
is the kernel mailing list.  I'm sorry, you want
debian-mentors@lists.debian.org or debian-devel@lists.debian.org, if
you're a Debian developer, or debian-kernel@lists.debian.org, if it's a
Debian kernel question (I'm sure they've lots of experience with not
using devfs by now), or possibly debian-user@lists.debian.org.  Your
custom packages should have source with them, or available, because they
should be GPL.  Do a little legwork.


> Yes I realise this change isn't out of the blue or anything, but it's
> in a 'stable' kernel.  Why bother calling 2.6 stable?  We may as well
> have stayed at 2.5 if this sort of thing is going to continue to be
> pulled.
>

What sort of thing?  Removal of long-deprecated subsystems that as much
as scream "Don't use me!  I'm going away!"?  That sort of thing?

How many 2.5 kernels could you actually run a production system on?  2.6
is remarkably version-interoperable for a kernel that a) works, b)
continues to get new code applied to it, c) is not dead, and d) everybody
complains about miserably.  It is remarkably stable in function.  Maybe
when we drop ata support for a bio implementation, we'll get a 2.7
series.  That might be a compatibility breaker.  But changing layers of
abstraction while leaving a perfectly functional migration path for users
of old code?  Dust off your brain and follow the well-blazed trail.

Please try to limit yourself to actual issues with actual kernels, and
title your emails something appropriate and non-incendiary.  And, do try
to keep up with current events.
 
> >
> > James
> >
> >
> 
> 
> 
> --
> "Genius might be described as a supreme capacity for getting its
> possessors
> into trouble of all kinds."
> -- Samuel Butler
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel"
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: Development tree, PLEASE?

[Matthew Frost]

--- Matthew Frost <artusemrys@sbcglobal.net> wrote:

> No.  Wrong.  If there're a whole grab bag, as you say, then you should
> post each, as a separate issue, possibly with consistent proposals for
> fixing them.  Follow protocol.  Posting a "The Kernel Is Falling" email
> gets people riled up, makes you look foolish, and gets nothing fixed. 
> Noise.  Send signal.  We'll wait.

Be it noted that you have clarified, and that the issue involves ARM and
trying to juggle solutions that have simpler alternatives; I just didn't
look low enough in the thread first.  My comments are superfluous at this
point.

Matt

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 20, 2006 7:19:58 PM -0800 Matthew Frost 
<artusemrys@sbcglobal.net> wrote:

>
>
> --- Matthew Frost <artusemrys@sbcglobal.net> wrote:
>
>> No.  Wrong.  If there're a whole grab bag, as you say, then you should
>> post each, as a separate issue, possibly with consistent proposals for
>> fixing them.  Follow protocol.  Posting a "The Kernel Is Falling" email
>> gets people riled up, makes you look foolish, and gets nothing fixed.
>> Noise.  Send signal.  We'll wait.
>
> Be it noted that you have clarified, and that the issue involves ARM and
> trying to juggle solutions that have simpler alternatives; I just didn't
> look low enough in the thread first.  My comments are superfluous at this
> point.

The thread in part diverged into three separate discussions more or less. 
I still have a big problem with how the development of the kernel is being 
done now, with a total lack of a stable branch.  Stable in my mind also 
means not a moving target for developers, nor maintainers.  Try maintaining 
a lot of very demanding applications that must run right so changes must 
always be fully tested before rolling out.  It makes it nearly impossible 
to do that when the kernel has no stable branch that's mostly bug fixes, 
instead any time a bug is discovered a full process must be started to make 
sure no new bugs in all the new code features, etc, that became present 
during the interim are not found.  It makes maintenance a real nightmare 
for atleast one environment in which I maintain production systems, and 
also makes it rather a bit more difficult in others too since changes must 
be vetted.  Not necessarily *all* of them, but when there's lots of changes 
it's hard to track whats 'interesting' and what doesn't affect one.  If 
there was/is a stable tree then when bugfixes come they are applied there, 
and one can upgrade along that tree with much less concern about things 
changing underfoot.

That's my root problem.  The devfs stuff is just ancillary and came about 
as examples of where I've been backed into a no win situation.  Yes, I know 
it was and is deprecated, fact is I didn't write all of the code that is 
the problem, and some of it I don't even have access to either.  Yes some 
of it is maintained by the distro, some by third parties, some by me.  But 
they're all being broken because we either throw out or try to return 
systems with these newer chipsets, or start forking and maintaining 
separate kernel trees.

Don't get me wrong, I understand the reasons, and I apologize for being so 
late to the party.

Over on my end I have to make a decision as to if I have the time to try 
to...promote/lead some sort of effort along these lines so that we can all 
benefit instead of just those willing to use and install RedHat or SuSE or 
Debian or Ubuntu or whatever distro.

I think this has gotten to beating a dead horse severely now, that may have 
already been dead when I walked into the room, and for that I apologize.

Re: Development tree, PLEASE?

[Lee Revell]

On Sat, 2006-01-21 at 00:22 -0700, Michael Loftis wrote:
> It makes maintenance a real nightmare 
> for atleast one environment in which I maintain production systems 

Why do you keep having to upgrade the kernel on production systems, if
the old kernel does what you need?

Lee

Re: Development tree, PLEASE?

[Jan Engelhardt]

>> Ok, so you agree that there was an ample warning that devfs is going
>> away... Now, what would be different if 2.8.0 released tomorrow
>> without devfs and your vendor would require you to build new Debian
>> installer and kernel?
>
> Because that would be expected.  That constitutes a major release, and should
> theoretically have had a development tree corresponding before it.

So let's rename 2.6.16 to 2.7.0, plus:

 - (implicitly with the *rename*) stop the 2.6.x series

 - never use 2.<even>.x again
   (some people still don't seem to get that <even> does not mean
   "stable" in the 2.4 sense)
   - or start 3.x with an overall new counting scheme

 - follow the current development model as usual

> I really understand atleast some of the reasons from the kernel development
> standpoint, and can see many really good reasons for running a development tree
> like this, and as a method of development I like and agree with it.
> However...for the general consumption there still needs to be some sort of
> stable target that can be used that's 'blessed' with that mark, and will get
> atleast some attention by developers for security updates and (mostly major)
> bugfixes, and that people can contribute these sorts of things to, probably
> with the proviso that they also contribute it to the mainline dev kernel maybe
> IE if you're going to add new supported device to 'stable' 2.6.16.x then you've
> got to add it to whatever the current 'dev' line is say 2.6.22 or whatever.
> The placing of the .'s is just symbolic.  It could be 2.6.x and 2.7.x just as
> in the past or it could be 3.0.0.x and 3.0.0+n


Jan Engelhardt
-- 

Re: Development tree, PLEASE?

[Jan Engelhardt]

>> I'd say the kernel tries damn hard at maintaining backwards
>> compatibility for userspace.
>> It tries less hard, but still makes a pretty good effort, for internal
>> APIs, but the real solution to the internal API churn is to get your
>> code merged as it'll then get updated automagically whenever something
>> changes - people who remove or change internals try very hard to also
>> update all (in-tree) users of the old stuff - take a look at when I
>> removed a small thing like verify_area() if you want an example.
>
> The only argument I have is one that won't fly at all here on LKML and that's
> about all the corporate sponsors the LK has that are also doing custom closed
> source modules that are only useful for their particular hardware.

It really does not fly. I run a "damn old" nvidia driver (1.0-4496)
with a little portforward work on a 2.6.15. It is slowly ceasing to
be perfect, but given that 4496 was originally only for 2.4, I'd say
that's good news. (It was first portforwarded by sh.nu to 2.6.4 or so,
until nvidia.com came up with 2.6 support on their own. I then took
the sh.nu port and pf'ed it on my own, and until now, the only things
that make slight gcc warnings are CONFIG_PM_LEGACY and the 4-level-pagetable
stuff. I'd say the API is stable long enough!)


Jan Engelhardt
-- 

Re: Development tree, PLEASE?

[Jesper Juhl]

On 1/21/06, Michael Loftis <mloftis@wgops.com> wrote:
>
> --On January 20, 2006 7:19:58 PM -0800 Matthew Frost
> <artusemrys@sbcglobal.net> wrote:
>
> > --- Matthew Frost <artusemrys@sbcglobal.net> wrote:
> >
> >> No.  Wrong.  If there're a whole grab bag, as you say, then you should
> >> post each, as a separate issue, possibly with consistent proposals for
> >> fixing them.  Follow protocol.  Posting a "The Kernel Is Falling" email
> >> gets people riled up, makes you look foolish, and gets nothing fixed.
> >> Noise.  Send signal.  We'll wait.
> >
> > Be it noted that you have clarified, and that the issue involves ARM and
> > trying to juggle solutions that have simpler alternatives; I just didn't
> > look low enough in the thread first.  My comments are superfluous at this
> > point.
>
> The thread in part diverged into three separate discussions more or less.
> I still have a big problem with how the development of the kernel is being
> done now, with a total lack of a stable branch.  Stable in my mind also
> means not a moving target for developers, nor maintainers.  Try maintaining
> a lot of very demanding applications that must run right so changes must
> always be fully tested before rolling out.  It makes it nearly impossible
> to do that when the kernel has no stable branch that's mostly bug fixes,
> instead any time a bug is discovered a full process must be started to make
> sure no new bugs in all the new code features, etc, that became present
> during the interim are not found.  It makes maintenance a real nightmare
> for atleast one environment in which I maintain production systems, and
> also makes it rather a bit more difficult in others too since changes must
> be vetted.  Not necessarily *all* of them, but when there's lots of changes
> it's hard to track whats 'interesting' and what doesn't affect one.  If
> there was/is a stable tree then when bugfixes come they are applied there,
> and one can upgrade along that tree with much less concern about things
> changing underfoot.
>
> That's my root problem.  The devfs stuff is just ancillary and came about
> as examples of where I've been backed into a no win situation.  Yes, I know
> it was and is deprecated, fact is I didn't write all of the code that is
> the problem, and some of it I don't even have access to either.  Yes some
> of it is maintained by the distro, some by third parties, some by me.  But
> they're all being broken because we either throw out or try to return
> systems with these newer chipsets, or start forking and maintaining
> separate kernel trees.
>
> Don't get me wrong, I understand the reasons, and I apologize for being so
> late to the party.
>
> Over on my end I have to make a decision as to if I have the time to try
> to...promote/lead some sort of effort along these lines so that we can all
> benefit instead of just those willing to use and install RedHat or SuSE or
> Debian or Ubuntu or whatever distro.
>

How about something along these lines :

What you want is a kernel that adheres to the rules set forth in
Documentation/stable_kernel_rules.txt , but you want it maintained for
longer than to the point where the next 2.6.x release happens - right?

So how about you pick, for example, the upcomming 2.6.16 kernel as
your stable target.
The first thing you do is join the -stable team to help review patches
going into 2.6.16.y and help find patches in the 2.6.17-rc kernels
that should go into -stable.
Then when 2.6.17 comes out the -stable team will normally abandon
2.6.16.y and move to 2.6.17 as the new -stable base, so now you have
to make a choice - you can either a) deside that 2.6.17 is an OK
upgrade for you and then the thing just reeats or b) you can deside
that you need to still stay with 2.6.16 for a bit longer.  In the case
of 'b' I'm sure noone would object to you keeing the 2.6.16.y -stable
tree going, so you keep backporting fixes to that (possibly some other
people will even help you with this) and continue to release 2.6.16.y
kernels.
At some point in time you deside to rebase on the latest 2.6.x kernel
and the whole thing repeats itself.

Help out the -stable team :)

--
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: Development tree, PLEASE?

[Ralf Baechle]

On Fri, Jan 20, 2006 at 08:17:40AM -0700, Michael Loftis wrote:

> 
> OK, I don't know abotu others, but I'm starting to get sick of this 
> unstable stable kernel.  Either change the statements allover that were 
> made that even-numbered kernels were going to be stable or open 2.7. 
> Removing devfs has profound effects on userland.  It's one thing to screw 
> with all of the embedded developers, nearly all kernel module developers, 
> etc, by changing internal APIs but this is completely out of hand.

Like devfs or not - but the elemination of devfs was announced ages ago
and anybody reading this list knew it for even much longer.  So this
example just serves to show that for some users no grace time is long
enough.

  Ralf

Re: Development tree, PLEASE?

[Alistair John Strachan]

On Saturday 21 January 2006 09:22, Jan Engelhardt wrote:
> >> I'd say the kernel tries damn hard at maintaining backwards
> >> compatibility for userspace.
> >> It tries less hard, but still makes a pretty good effort, for internal
> >> APIs, but the real solution to the internal API churn is to get your
> >> code merged as it'll then get updated automagically whenever something
> >> changes - people who remove or change internals try very hard to also
> >> update all (in-tree) users of the old stuff - take a look at when I
> >> removed a small thing like verify_area() if you want an example.
> >
> > The only argument I have is one that won't fly at all here on LKML and
> > that's about all the corporate sponsors the LK has that are also doing
> > custom closed source modules that are only useful for their particular
> > hardware.
>
> It really does not fly. I run a "damn old" nvidia driver (1.0-4496)
> with a little portforward work on a 2.6.15. It is slowly ceasing to
> be perfect, but given that 4496 was originally only for 2.4, I'd say
> that's good news. (It was first portforwarded by sh.nu to 2.6.4 or so,
> until nvidia.com came up with 2.6 support on their own. I then took
> the sh.nu port and pf'ed it on my own, and until now, the only things
> that make slight gcc warnings are CONFIG_PM_LEGACY and the
> 4-level-pagetable stuff. I'd say the API is stable long enough!)

Myself and Christian Zander were the original authors of the port to 2.6, well 
before 2.6.0 was released. I think it's wrong to claim that the API changes 
between 2.4 and 2.6 were trivial or limited, we made a significant number of 
changes to the driver in almost every subsystem -- memory management, AGP 
handling, devfs support, module support, bh versus tasklets, etc..

Also if you look even today at Makefile.kbuild, there's quite a significant 
amount of work required to get it work with both 2.4 and 2.6 (the 2.6 code is 
obviously simpler). To top it off, supporting these "vendor kernels" 
mentioned in this thread also requires many pre-compilation checks.

I'm of the opinion that the kernel API should not be stable, but let's please 
not pretend that it is. That simply is not the case. For vendors, maintaining 
support for literally years of kernels is a challenge.

-- 
Cheers,
Alistair.

'No sense being pessimistic, it probably wouldn't work anyway.'
Third year Computer Science undergraduate.
1F2 55 South Clerk Street, Edinburgh, UK.

Re: Development tree, PLEASE?

[Jan Engelhardt]

>
>Myself and Christian Zander were the original authors of the port to 2.6, well 
>before 2.6.0 was released. I think it's wrong to claim that the API changes 

Yes, I took the one that was/is available on http://sh.nu/nvidia/ 
and from _then_ (not early - I do not claim 2.4->2.6 portage) did my own 
stuff.


Jan Engelhardt
-- 

Re: Development tree, PLEASE?

[Horst von Brand]

Michael Loftis <mloftis@wgops.com> wrote:

[...]

> The thread in part diverged into three separate discussions more or
> less. I still have a big problem with how the development of the
> kernel is being done now, with a total lack of a stable branch.
> Stable in my mind also means not a moving target for developers, nor
> maintainers.  Try maintaining a lot of very demanding applications
> that must run right so changes must always be fully tested before
> rolling out.  It makes it nearly impossible to do that when the kernel
> has no stable branch that's mostly bug fixes, instead any time a bug
> is discovered a full process must be started to make sure no new bugs
> in all the new code features, etc, that became present during the
> interim are not found.  It makes maintenance a real nightmare for
> atleast one environment in which I maintain production systems, and
> also makes it rather a bit more difficult in others too since changes
> must be vetted.  Not necessarily *all* of them, but when there's lots
> of changes it's hard to track whats 'interesting' and what doesn't
> affect one.  If there was/is a stable tree then when bugfixes come
> they are applied there, and one can upgrade along that tree with much
> less concern about things changing underfoot.

What has changed underfoot you? If it affects userspace, I gather that
would be considered a bug (or you doing nasty things you shouldn't have
done in the first place). 

> That's my root problem.  The devfs stuff is just ancillary and came
> about as examples of where I've been backed into a no win situation.

No. /You/ decided to paint yourself into a tight corner there. Ad did it
carefully, over a couple of years.

[...]

> Over on my end I have to make a decision as to if I have the time to
> try to...promote/lead some sort of effort along these lines so that we
> can all benefit instead of just those willing to use and install
> RedHat or SuSE or Debian or Ubuntu or whatever distro.

Ask the Red Hat people how much it costs them to keep the stuff up to date
for RHEL... better go for 2.4.x or so.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: Development tree, PLEASE?

[Johan Kullstam]

tab@snarc.org (Vincent Hanquez) writes:

> On Fri, Jan 20, 2006 at 02:27:50PM -0500, Ben Collins wrote:
> > So put me in for +1 on the current development model.
> 
> I'ld like to also +1 the current development model.
> 
> However I think the 2.6.X should become 2.X; the 6 is starting to be
> meaningless, and it's not going to be upgraded ever with the current
> development model.

Why not just drop the "2"?  It's not like the "2" is going anywhere
with current or even with the past development models.  2.X.Y has
already been used (for X = 0-6 and modest Y), so 6.16.1 could be used
instead of 2.6.16-rc1.

> this leave 2.X.Y "namespace" to the current stable team (same
> development model as the 2.6.X.Y).

-- 
Johan KULLSTAM

Re: Development tree, PLEASE?

[Sven-Haegar Koch]

On Sat, 21 Jan 2006, Lee Revell wrote:

> On Sat, 2006-01-21 at 00:22 -0700, Michael Loftis wrote:
>> It makes maintenance a real nightmare
>> for atleast one environment in which I maintain production systems
>
> Why do you keep having to upgrade the kernel on production systems, if
> the old kernel does what you need?

But it is missing all security updates.

What I am currently doing to workaround this problem:

- using Debian Sarge on my production servers as a base
   (good packages, but kernel is just too old)
- Kernel 2.6.12 from Ubuntu Breezy (taken as source, not binary packages)

This way I have at least a working kernel (2.6.8 does not work on my newer 
boxes) and the security updates from Ubuntu, getting kernel updates with 
only little changes and low update-risks.

Mainstream kernel is just unusable when you don't have the time to verify
the lots of changes in production environments.

c'ya
sven

-- 

The Internet treats censorship as a routing problem, and routes around it.
(John Gilmore on http://www.cygnus.com/~gnu/)

Re: Development tree, PLEASE?

[Lee Revell]

On Sat, 2006-01-21 at 22:56 +0100, Sven-Haegar Koch wrote:
> On Sat, 21 Jan 2006, Lee Revell wrote:
> 
> > On Sat, 2006-01-21 at 00:22 -0700, Michael Loftis wrote:
> >> It makes maintenance a real nightmare
> >> for atleast one environment in which I maintain production systems
> >
> > Why do you keep having to upgrade the kernel on production systems, if
> > the old kernel does what you need?
> 
> But it is missing all security updates.
> 
> What I am currently doing to workaround this problem:
> 
> - using Debian Sarge on my production servers as a base
>    (good packages, but kernel is just too old)
> - Kernel 2.6.12 from Ubuntu Breezy (taken as source, not binary packages)
> 
> This way I have at least a working kernel (2.6.8 does not work on my newer 
> boxes) and the security updates from Ubuntu, getting kernel updates with 
> only little changes and low update-risks.
> 
> Mainstream kernel is just unusable when you don't have the time to verify
> the lots of changes in production environments.

You just illustrated perfectly why the new development model is needed.

If 2.6.8 does not even work on newer machines, then obviously the rapid
pace of development is needed in order to support new hardware.  You
can't have a kernel that supports the latest and greatest hardware
without the possibility of introducing bugs.

Lee

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 21, 2006 5:18:01 PM -0500 Lee Revell <rlrevell@joe-job.com> 
wrote:


> You just illustrated perfectly why the new development model is needed.
>
> If 2.6.8 does not even work on newer machines, then obviously the rapid
> pace of development is needed in order to support new hardware.  You
> can't have a kernel that supports the latest and greatest hardware
> without the possibility of introducing bugs.

I don't feel that statement is true in all cases.  It's true in a lot of 
cases yes, but sometimes 'support' is really simply a matter of techinga 
module one more PCI ID.  Or adding in a few lines of code for a different 
PHY in the case of an ethernet adapter/MAC.  You also don't need to change 
say the queue elevator mechanism to support a new SATA chipset.  What the 
complaint is from production systems is the fact that in many many cases 
for new hardware support all that's needed is the little bit of code way 
out on the edge, without changing anything else.  While I am of the opinion 
new hardware support in a maintenance/bugfix/stabel kernel is a gray area, 
I can see where there are many times where it could be done, without 
introducing excessive change.

Right now the most obvious is the udev/devfs problem, at some point you're 
forced to lose that, that's fine, but there's nowhere to go for bugfixes. 
That's *ONE* case.  There WILL be more/others, possibly ones that are more 
disruptive.  Yes distro's/etc should help, but some of us aren't using 
distro's.  Which is why I started the thread is to try to get a community 
based longer lived stable tree.  Hopefully one that distro's would also 
use, and contribute bugfixes and patches to, much the same way as we have 
had in the past, but without directly taking the mainline/core team since 
they've decided (and I don't begrudge them this at all) that it is too much 
work to continue maintaining a stable tree, as well as developing, the 
kernel.

If possible I'd like Sven (even privately) to summarize does not work in a 
little bit clearer way. IE what bits don't work, or does it just completely 
fail to boot?  TIA!  I think he and I are seeing the same problems and are 
on the same page with this overall larger issue.

>
> Lee
>
>



--
"Genius might be described as a supreme capacity for getting its possessors
into trouble of all kinds."
-- Samuel Butler

Re: Development tree, PLEASE?

[Lee Revell]

On Sat, 2006-01-21 at 15:40 -0700, Michael Loftis wrote:
> I don't feel that statement is true in all cases.  It's true in a lot
> of cases yes, but sometimes 'support' is really simply a matter of
> techinga module one more PCI ID.  

That might be true for server class hardware where the vendors care
about the stability of the hardware platform, but for modern desktop
stuff like sound cards it's never that simple.  Desktop class hardware
is getting dumber and dumber all the time as more features are pushed
into software which makes adding support for new devices tricky, and new
devices are introduced constantly.  Sometimes they'll even ship 2 models
with the same PCI IDs but a different chipset, so you have to use some
other kludge to know what to do.  Etc.

Lee

Re: Development tree, PLEASE?

[Bernd Petrovitsch]

On Sat, 2006-01-21 at 15:40 -0700, Michael Loftis wrote:
[...]
>         Which is why I started the thread is to try to get a community 
> based longer lived stable tree.  Hopefully one that distro's would also 

AFAICS (ATM), you have exactly three possibilities:
- Start and maintain this "community based longer lived stable tree".
- Pay someone to do it.
  If you can be more precise, what you really want I can write you an
  offer.
- Stop whining that others are not doing the workfor you (i.e. *your
  work*) for free.

	Bernd
-- 
Firmix Software GmbH                   http://www.firmix.at/
mobil: +43 664 4416156                 fax: +43 1 7890849-55
          Embedded Linux Development and Services

Re: Development tree, PLEASE?

[Bernd Petrovitsch]

On Sat, 2006-01-21 at 17:47 -0500, Lee Revell wrote:
> On Sat, 2006-01-21 at 15:40 -0700, Michael Loftis wrote:
> > I don't feel that statement is true in all cases.  It's true in a lot
> > of cases yes, but sometimes 'support' is really simply a matter of
> > techinga module one more PCI ID.  
> 
> That might be true for server class hardware where the vendors care
> about the stability of the hardware platform, but for modern desktop
> stuff like sound cards it's never that simple.  Desktop class hardware
> is getting dumber and dumber all the time as more features are pushed
> into software which makes adding support for new devices tricky, and new
> devices are introduced constantly.  Sometimes they'll even ship 2 models
> with the same PCI IDs but a different chipset, so you have to use some
> other kludge to know what to do.  Etc.

And the more the development head proceeds from the "stable" one, the
more work is it to "backport" some hardware driver IMHO. Especially if
"stability" is a primary concern.

	Bernd
-- 
Firmix Software GmbH                   http://www.firmix.at/
mobil: +43 664 4416156                 fax: +43 1 7890849-55
          Embedded Linux Development and Services

Re: Development tree, PLEASE?

[Lee Revell]

On Sat, 2006-01-21 at 15:40 -0700, Michael Loftis wrote:
> I don't feel that statement is true in all cases.  It's true in a lot
> of cases yes, but sometimes 'support' is really simply a matter of
> techinga module one more PCI ID.  Or adding in a few lines of code for
> a different PHY in the case of an ethernet adapter/MAC.  You also
> don't need to change say the queue elevator mechanism to support a new
> SATA chipset.  What the complaint is from production systems is the
> fact that in many many cases for new hardware support all that's
> needed is the little bit of code way out on the edge, without changing
> anything else.  

In order to "support" AMD X2 systems, it was necessary to revamp the
kernel's internal timekeeping.  How are we expected to deal with vendors
who break backwards compatibility on a deep level like this?

So basically a "stable kernel" means no new hardware support, which
basically means it's dead from the development POV - who would want to
work on such a thing?

Lee

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 21, 2006 11:51:28 PM +0100 Bernd Petrovitsch <bernd@firmix.at> 
wrote:

> And the more the development head proceeds from the "stable" one, the
> more work is it to "backport" some hardware driver IMHO. Especially if
> "stability" is a primary concern.

Yes, I realise all of this.  But everyone seems to get this damned 
territorial attitude that I want to see kernel development stopped, quite 
the opposite.  All I want to see is a stable target for certain windows of 
time.  So that way when bugs are fixed that are trivial there's a place to 
go without upgrading scads of userland stuff or worrying about lots of 
unrelated change.

>
> 	Bernd
> --
> Firmix Software GmbH                   http://www.firmix.at/
> mobil: +43 664 4416156                 fax: +43 1 7890849-55
>           Embedded Linux Development and Services
>
>
>
>



--
"Genius might be described as a supreme capacity for getting its possessors
into trouble of all kinds."
-- Samuel Butler

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 21, 2006 6:03:01 PM -0500 Lee Revell <rlrevell@joe-job.com> 
wrote:

> On Sat, 2006-01-21 at 15:40 -0700, Michael Loftis wrote:
>> I don't feel that statement is true in all cases.  It's true in a lot
>> of cases yes, but sometimes 'support' is really simply a matter of
>> techinga module one more PCI ID.  Or adding in a few lines of code for
>> a different PHY in the case of an ethernet adapter/MAC.  You also
>> don't need to change say the queue elevator mechanism to support a new
>> SATA chipset.  What the complaint is from production systems is the
>> fact that in many many cases for new hardware support all that's
>> needed is the little bit of code way out on the edge, without changing
>> anything else.
>
> In order to "support" AMD X2 systems, it was necessary to revamp the
> kernel's internal timekeeping.  How are we expected to deal with vendors
> who break backwards compatibility on a deep level like this?
>
> So basically a "stable kernel" means no new hardware support, which
> basically means it's dead from the development POV - who would want to
> work on such a thing?

That's why there's a maintenance/stable branch of most every single 
project, commercial or otherwise, and a development branch.  Development 
for new hardware continues, and for people who need these pieces of 
hardware which require major changes to work, then this much more limited 
set of users can take the rest of the issues that follow with using a dev 
kernel, until the stable branch moves up to/off/after the point at which 
the development branch got support for their new hardware.

A *lot* of us are using Linux for servers or other things that don't change 
every month.

And I'm not seeing/saying this sort of thing would stick forever, but a '6 
month cycle' or something of that nature.  Partly because of this I don't 
forsee myself having time to really start work on this for another month or 
two since I have to go into devel/bunker and get things working now at the 
demand of other entities than myself.

This thread has shown that there is desire for such a thing atleast by a 
few.  I'm just sure it's not a one man job, I've also been given a pointer 
that there is a stable team and I've yet to have time to go in that 
direction (really stirred the ants nest with this one).

>
> Lee
>
>



--
"Genius might be described as a supreme capacity for getting its possessors
into trouble of all kinds."
-- Samuel Butler

Re: Development tree, PLEASE?

[Theodore Ts'o]

On Sun, Jan 22, 2006 at 01:57:19AM -0700, Michael Loftis wrote:
> 
> Yes, I realise all of this.  But everyone seems to get this damned 
> territorial attitude that I want to see kernel development stopped, quite 
> the opposite.  All I want to see is a stable target for certain windows of 
> time.  So that way when bugs are fixed that are trivial there's a place to 
> go without upgrading scads of userland stuff or worrying about lots of 
> unrelated change.

A stable target is trivial; you just don't make any changes to it.
2.6.10 is a stable target. However, the moment you start wanting
security fixes, and support for new hardware, while still having a
"stable target", this is where life gets difficult. 

The disconnect seems to be on how hard this is perceived to be; you
seem to be focusing on the trivial cases, where all you have to do is
add a new PCI ID to a driver's white list, for example.  Everyone else
is focusing on all of the horror stories where in order to support new
hardware, major pieces of core kernel functionality had to be ripped
apart and rearchitected in order to support said new hardware, or the
problems where people only develop a fix to one specific kernel
version, and no one else bothers to forward-port or back-port the fix
to other kernel versions.

Obviously the truth lies somewhere in between these two extremes, but
I think there are some indicators that might tend to show that those
who think you are wrong might have a point.

a) If this was so easy, someone would have done it by now ---
especially those maintaining distro kernels of one kind or another:
i.e., Red Hat, SuSE, Debian, Ubuntu, etc.

b) As a related argument, "if you think this is so easy, why don't you
try it yourself?"

c) We have tried to do it in the past, i.e., with 2.4, and it was
pretty clear that in the long run, it didn't work well at all.

Maybe you think that "certain windows of time" is would make the
problem tractable if it were shorter than 2.4 was in practice, but
longer than the current 2.6.x.y stable series (for example).

I still think that in the long run, if you want to be able to support
new hardware, it is inevitable that you will be disappointed.  Yes
sometimes it only requires a new PCI ID --- until you run into the
time when it requires a major roto-tilling of the main project.

						- Ted

Re: Development tree, PLEASE?

[Bernd Petrovitsch]

On Sun, 2006-01-22 at 01:57 -0700, Michael Loftis wrote:
> --On January 21, 2006 11:51:28 PM +0100 Bernd Petrovitsch <bernd@firmix.at> 
> wrote:
> 
> > And the more the development head proceeds from the "stable" one, the
> > more work is it to "backport" some hardware driver IMHO. Especially if
> > "stability" is a primary concern.
> 
> Yes, I realise all of this.  But everyone seems to get this damned 
> territorial attitude that I want to see kernel development stopped, quite 

Not at all.

> the opposite.  All I want to see is a stable target for certain windows of 
> time.  So that way when bugs are fixed that are trivial there's a place to 

You already have it - with the old and with the new development model.
*Your* problem is that the window of time with the new model is too
short IYHO.

> go without upgrading scads of userland stuff or worrying about lots of 
> unrelated change.

Yes, we all know *your* problem and you want dozens of people to work a
lot *only* for you and for free.
Other solutions to *your* problem can be found in other mails in this
thread.

	Bernd
-- 
Firmix Software GmbH                   http://www.firmix.at/
mobil: +43 664 4416156                 fax: +43 1 7890849-55
          Embedded Linux Development and Services

Re: Development tree, PLEASE?

[Bernd Petrovitsch]

On Sun, 2006-01-22 at 02:03 -0700, Michael Loftis wrote:
[...]
> A *lot* of us are using Linux for servers or other things that don't change 
> every month.
> 
> And I'm not seeing/saying this sort of thing would stick forever, but a '6 
> month cycle' or something of that nature.  Partly because of this I don't 

You are welcome to start such a thing.

	Bernd
-- 
Firmix Software GmbH                   http://www.firmix.at/
mobil: +43 664 4416156                 fax: +43 1 7890849-55
          Embedded Linux Development and Services

Re: Development tree, PLEASE?

[Arjan van de Ven]

> And I'm not seeing/saying this sort of thing would stick forever, but a '6 
> month cycle' or something of that nature.  

Guess how long the current -stable tree is maintained... yes that is
right, just about 6 months give or take a week or two.

Ergo, your point is moot.

Re: Development tree, PLEASE?

[Lee Revell]

On Sun, 2006-01-22 at 02:03 -0700, Michael Loftis wrote:
> Development for new hardware continues, and for people who need these
> pieces of hardware which require major changes to work, then this much
> more limited set of users can take the rest of the issues that follow
> with using a dev kernel, until the stable branch moves up to/off/after
> the point at which the development branch got support for their new
> hardware.
> 
> A *lot* of us are using Linux for servers or other things that don't
> change every month. 

Well, like it or not, desktop, embedded, and HPC are where the action
is, and the new development model reflects that.

The server war is over, and we won, back in the 2.4 era.

Lee 

Re: Development tree, PLEASE?

[Daniel Barkalow]

On Sun, 22 Jan 2006, Michael Loftis wrote:

> Yes, I realise all of this.  But everyone seems to get this damned 
> territorial attitude that I want to see kernel development stopped, 
> quite the opposite.  All I want to see is a stable target for certain 
> windows of time.  So that way when bugs are fixed that are trivial 
> there's a place to go without upgrading scads of userland stuff or 
> worrying about lots of unrelated change.

I believe that, if you want to maintain a 2.6.13.y (for example) tree 
after the -stable team has moved on, you'd be perfectly welcome, and could 
probably even do it on kernel.org. It might not even be that hard to get 
the necessary patches, given that -stable sees all of the long-standing 
stability/security bugs (so you can watch that list for ones you should 
include patches for), and the regressions will probably mostly be fixed 
before you get the series.

I think that the reason that nobody's done this already isn't that it 
would be very difficult, but that distributions don't actually see a value 
in using old kernel series and are happy with -stable. If you have a 
reason to stick with a series longer, it might be worth the trouble to 
you.

	-Daniel
*This .sig left intentionally blank*

Re: Development tree, PLEASE?

[Vincent Hanquez]

On Sat, Jan 21, 2006 at 01:29:44PM -0500, Johan Kullstam wrote:
> Why not just drop the "2"?  It's not like the "2" is going anywhere
> with current or even with the past development models.  2.X.Y has
> already been used (for X = 0-6 and modest Y), so 6.16.1 could be used
> instead of 2.6.16-rc1.

Just want to make clear, I wasn't taking about reusing 2.0, 2.1 ..
but just continue from 2.6 as the following example:
   
   2.6.16   => 2.7.0
   2.6.16.1 => 2.7.1
   2.6.16.2 => 2.7.2
   2.6.17   => 2.8.0
   2.6.18   => 2.9.0
   ...

The development model stays exactly the same:
i.e. 2.X.0 kernels are released every ~N weeks by Linus.

-- 
Vincent Hanquez

Re: Development tree, PLEASE?

[Bob Copeland]

On 21 Jan 2006 13:29:44 -0500, Johan Kullstam <kullstj-ml@comcast.net> wrote:
> Why not just drop the "2"?  It's not like the "2" is going anywhere
> with current or even with the past development models.  2.X.Y has
> already been used (for X = 0-6 and modest Y), so 6.16.1 could be used
> instead of 2.6.16-rc1.

Please no.  Let's not follow Sun off of that particular version number cliff.

Bob

Re: Development tree, PLEASE?

[Nix]

On 22 Jan 2006, Bernd Petrovitsch gibbered uncontrollably:
> On Sun, 2006-01-22 at 02:03 -0700, Michael Loftis wrote:
> [...]
>> A *lot* of us are using Linux for servers or other things that don't change 
>> every month.
>> 
>> And I'm not seeing/saying this sort of thing would stick forever, but a '6 
>> month cycle' or something of that nature.  Partly because of this I don't 
> 
> You are welcome to start such a thing.

Besides, the distinction between a 6 month cycle and a 3 month cycle isn't
all that great, and we have a 3 month cycle *now*.

Backporting the security-fix parts of 2.6.15-stable to 2.6.14 isn't
likely to be killingly difficult, especially now that 2.6.14 rollup
-stable patches are being done.

-- 
`Everyone has skeletons in the closet.  The US has the skeletons
 driving living folks into the closet.' --- Rebecca Ore

Re: Development tree, PLEASE?

[Lee Revell]

On Wed, 2006-01-25 at 21:30 +0000, Nix wrote:
> Besides, the distinction between a 6 month cycle and a 3 month cycle
> isn't all that great, and we have a 3 month cycle *now*.
> 

If you want to be a viable desktop OS which requires supporting the
latest hardware 6 months is too slow.  Many vendors of desktop junk come
out with a new card every 3-6 months.  Users don't want to wait a year
for their favorite distro to support their new hardware.

Lee

Re: Development tree, PLEASE?

[Nix]

On Wed, 25 Jan 2006, Lee Revell announced authoritatively:
> On Wed, 2006-01-25 at 21:30 +0000, Nix wrote:
>> Besides, the distinction between a 6 month cycle and a 3 month cycle
>> isn't all that great, and we have a 3 month cycle *now*.
> 
> If you want to be a viable desktop OS which requires supporting the
> latest hardware 6 months is too slow.  Many vendors of desktop junk come
> out with a new card every 3-6 months.  Users don't want to wait a year
> for their favorite distro to support their new hardware.

Indeed. I'm not sure if any distros actually *do* release every three
months, but there tends to be *a* release of some major distro every
three months, and some early adopters really do bounce from distro
to distro whenever one of those releases happens so they can get at
the card they bought last week. :)

-- 
`Everyone has skeletons in the closet.  The US has the skeletons
 driving living folks into the closet.' --- Rebecca Ore

Re: Development tree, PLEASE?

[Bernd Petrovitsch]

On Wed, 2006-01-25 at 22:12 +0000, Nix wrote:
[...]
> Indeed. I'm not sure if any distros actually *do* release every three

Gentoo would probably qualify for that.
Debian, FC, SusE not.
Others I don't know enough ....

	Bernd
-- 
Firmix Software GmbH                   http://www.firmix.at/
mobil: +43 664 4416156                 fax: +43 1 7890849-55
          Embedded Linux Development and Services

Re: Development tree, PLEASE?

[Jan Engelhardt]

>[...]
>> Indeed. I'm not sure if any distros actually *do* release every three
>
>Gentoo would probably qualify for that.

>Debian, FC, SusE not.

"Advanced users" can pick a SUSE KOTD.



Jan Engelhardt
-- 

Re: Development tree, PLEASE?

[Bernd Petrovitsch]

On Thu, 2006-01-26 at 22:12 +0100, Jan Engelhardt wrote:
> >[...]
> >> Indeed. I'm not sure if any distros actually *do* release every three
> >
> >Gentoo would probably qualify for that.
> 
> >Debian, FC, SusE not.
> 
> "Advanced users" can pick a SUSE KOTD.
                                   ^^^^
I assume this means "Kernel of the day".
First I didn't know of this KOTD thing.
And second, I don't know if this fits the OPs definitioen of "release".
For gentoo, I interpreted/defined "release" as "the thing you with
`emerge update` without ~x86 overriding or similar".

	Bernd
-- 
Firmix Software GmbH                   http://www.firmix.at/
mobil: +43 664 4416156                 fax: +43 1 7890849-55
          Embedded Linux Development and Services

Re: Development tree, PLEASE?

[David Weinehall]

[snip]

So, let's summarise what you've been saying in this thread so far:

o You want advance warning of API changes, but when you get them
  (devfs, for instance), you ignore them and complain anyway -- check

o You want security fixes and only minor other fixes (done magically
  by someone else as you're not willing to pay for it, nor are you
  willing to help yourself), for at least 6 months, but you ignore
  the existance of the 2.6.x.y kernel series, which does exactly
  that -- check

o You think that 2.4.x isn't supporting enough new hardware,
  and yet you claim that adding new PCI ID:s is enough to add
  support for new hardware in most cases -- check

o You're going on and on about API breakage between kernel and
  userspace, yet the only example you keep repeating is devfs -- check

So far, I'd say you're just trolling.  Please calm down, *breathe*,
and start reading what people actually respond to you, think it
through, and consider if maybe, just maybe, there might be more sense
in their opinions than in yours.  And maybe, just maybe, people that
spend a lot of their spare time (or work time, for that matter) to
give you for free (and FREE) the best kernel there is, deserve a
bit more than your whining.

Or in short:

"Don't complain, contribute!"


Regards: David Weinehall
-- 
 /) David Weinehall <tao@acc.umu.se> /) Northern lights wander      (\
//  Maintainer of the v2.0 kernel   //  Dance across the winter sky //
\)  http://www.acc.umu.se/~tao/    (/   Full colour fire           (/

Re: Development tree, PLEASE?

[Michael Loftis]

--On February 2, 2006 1:16:53 PM +0100 David Weinehall <tao@acc.umu.se> 
wrote:

> [snip]
>
> So, let's summarise what you've been saying in this thread so far:
>
> o You want advance warning of API changes, but when you get them
>   (devfs, for instance), you ignore them and complain anyway -- check

The problem is that there's no more stable kernel first, and secondly that 
there's not much if any pointers abotu the change.  People complained I 
didn't have specific problems to solve, I don't CARE abotu any specific 
problems.  I could hardly give a rats ass about devfs.  It's the whole new 
development model that's the problem, and will only get worse for the types 
of companies who I work with who normally right now support Linux 
development.  I only brought up devfs because certain members of the 
community can't see past bootstrings to the bigger issue.

>
> o You want security fixes and only minor other fixes (done magically
>   by someone else as you're not willing to pay for it, nor are you
>   willing to help yourself), for at least 6 months, but you ignore
>   the existance of the 2.6.x.y kernel series, which does exactly
>   that -- check

There's noone out there that does that, I'd LOVE to be able to pay for it 
and not have to do it myself.  RedHat kernels don't work on most of our 
gear, and RH, up to and including fedora core, and centos have some 'great' 
issues, like the listener processes for Apache and MySQL using up *ALL* of 
the system CPU when *NOTHING* is happening.  We've tried to track it down, 
it's gotten to where we just don't care and we just don't deploy RedHat 
anymore.  SuSE's kernels suffer the same problem of too many patches I 
mentioned before for totally unrelated, non-security things.

Further, I'm not the only one.  I'm the only one who has enough asbestos to 
jump onto LKML and say it because they all know that it's completely 
hostile in here when someone brings up something that looks like a major 
issue.

> o You think that 2.4.x isn't supporting enough new hardware,
>   and yet you claim that adding new PCI ID:s is enough to add
>   support for new hardware in most cases -- check

No I don't i said in MANY or atleast SOME.  further 2.4 is supposedly DEAD 
anyway.

> o You're going on and on about API breakage between kernel and
>   userspace, yet the only example you keep repeating is devfs -- check
>
> So far, I'd say you're just trolling.  Please calm down, *breathe*,
> and start reading what people actually respond to you, think it
> through, and consider if maybe, just maybe, there might be more sense
> in their opinions than in yours.  And maybe, just maybe, people that
> spend a lot of their spare time (or work time, for that matter) to
> give you for free (and FREE) the best kernel there is, deserve a
> bit more than your whining.

And it's only been the best because there's been a way for people to sue 
it, get security patches even teh occasional new hardware support when it's 
not disruptive.  Now that entire development model has changed into 
something that noone uses because it doesn't' work for a project needing a 
stable tree, such as a kernel.  It seems most people here just can't 
understand how it can become a problem unfortunately because they can't 
really see the big picture.  Everyone wants to take one little piece and go 
hey hey see that's not a problem and then self satisfied at their attacking 
and dismissal think they've solved the problem.  This is not a single 
problem issue.

>
> Or in short:
>
> "Don't complain, contribute!"

I'm damned sick of the number of people who just *ATTACK* people who 
contribute.  Constructive criticism is a form of contribution, feedback if 
those words are too big for some to understand.  Because of the development 
model changes there are projects not going to use Linux at several 
companies that I work for contracting.  Because there is no way that any 
single entity can look at 4+MB of compressed code changes and be able to be 
even remotely sure that the kernel is going to work, and that's been the 
case.  That combined with the rapid API changes, and noone is developing a 
long lived stable kernel anymore means that commercial support of this OS 
is being lost.  I'm in an odd situation where because of NDAs and etc. I 
can not disclose any real details about the commercial backers, but I'm 
sure they're not the only ones, and probably much more important ones are 
getting frustrated.

Informational input can and often is as valuable as code.  Getting someone 
to think of something they hadn't thought of can save that person or the 
whole group lots of effors.

So, if you don't have anything USEFUL to retort back, shut up.  I'm getting 
sick of hearing the people who can't contribute *ANYTHING NEW* to the 
conversation and I'm in a very short mood.

The ... attitude and atmosphere here on LKML when someone brings up 
something even slightly controversial is detrimental.  I know all of you 
mean well.  But really.  If you're not contributing then you can stay quiet 
just as well as the person you're complaining at.

This thread has been closed for what?   A week now?  I'm working on trying 
to get the systems that are currently my big problems going, and after that 
then I can focus more attention on the points I've brought up earlier.  I'm 
only one person and just because I can't act immediately to do something 
does not mean I won't.  Any of us who has an extremely busy day job sure 
can understand that statement.

Re: Development tree, PLEASE?

[Dave Jones]

On Thu, Feb 02, 2006 at 11:25:24AM -0700, Michael Loftis wrote:

 > >o You want security fixes and only minor other fixes (done magically
 > >  by someone else as you're not willing to pay for it, nor are you
 > >  willing to help yourself), for at least 6 months, but you ignore
 > >  the existance of the 2.6.x.y kernel series, which does exactly
 > >  that -- check
 > 
 > There's noone out there that does that, I'd LOVE to be able to pay for it 
 > and not have to do it myself.  RedHat kernels don't work on most of our 
 > gear

Specifics?  The patches we carry in Fedora aren't very system-specific,
so any failure to boot there would likely be a problem on mainline.
The "but it works on $otherdistro" response that seems to be so popular
these days is time after time proven due to be because $otherdistro is
shipping an older kernel, and hasn't hit that particular bug yet.

 > , and RH, up to and including fedora core, and centos have some 'great' 
 > issues, like the listener processes for Apache and MySQL using up *ALL* of 
 > the system CPU when *NOTHING* is happening.

How did you determine this is a kernel bug?  Did you file a bugzilla report on this?

 > We've tried to track it down, 
 > it's gotten to where we just don't care and we just don't deploy RedHat 
 > anymore.  SuSE's kernels suffer the same problem of too many patches I 
 > mentioned before for totally unrelated, non-security things.

You can't complain about 'too many patches' in a distro kernel these days.
Any distro that ships a stock vanilla kernel is a distro that has
known oopsable drivers, features that don't work as expected, 
won't boot on certain hardware, and other general flakyness.

In the current FC4 2.6.15.2 based update..

- 47 bug fix patches. Shipping without these isn't an option.
- 24 'deviation' patches, where we add some not-yet-upstream feature
  or rh-specific feature. (Xen, Execshield, signed modules, restricted /dev/mem etc)
  [note, not 1 diff per feature, some features are multiple patches]
- 21 debugging patches. (Enabling extra output in certain bad situations etc)
- 3 convenient 'make the buildsys life easier' patches
- the remainder are other 'nice to haves' backported from 2.6.16rc

At the absolute minimum, we'd need to carry those 47 bugfixes.
Some of these get pushed to -stable, some aren't considered enough
of a problem, so they'll sit there until I rebase to a .16

We have this mentality in certain circles of "I don't want any
changes in my distro kernel, oh, except for ones that I want".
The problem is when >1 person wants patches to make their systems
run, the result is a pretty large collection of patches.

If you want a kernel with a limited set of patches, the only answer
is 'build your own', but don't complain about vendors doing the
only thing that they can do that they've been doing for the last
10-15 years -- Trying to make a single kernel image work on
as many platforms as possible with the smallest amount of pain.
(And 'new' development model hasn't changed a damn thing in this regard,
 it's always been a challenge).

		Dave

Re: Development tree, PLEASE?

[Willy Tarreau]

On Thu, Feb 02, 2006 at 11:25:24AM -0700, Michael Loftis wrote:
> >o You think that 2.4.x isn't supporting enough new hardware,
> >  and yet you claim that adding new PCI ID:s is enough to add
> >  support for new hardware in most cases -- check
> 
> No I don't i said in MANY or atleast SOME.  further 2.4 is supposedly 
> DEAD anyway.

Because it does not get marketting people's attention anymore does not
mean it's dead, the opposite. They've done their work at selling it.
Now it runs at many places. Distro vendors like Red Hat will still
support it for a few years. Mission-critical systems have not moved to
2.6 yet ! Any I'm sure that David could give you amazing examples of
critical systems reliably running on 2.0 with hundreds of days of
uptime !

> I'm damned sick of the number of people who just *ATTACK* people who 
> contribute.  Constructive criticism is a form of contribution, feedback 
> if those words are too big for some to understand.  Because of the 
> development model changes there are projects not going to use Linux at 
> several companies that I work for contracting.  Because there is no way 
> that any single entity can look at 4+MB of compressed code changes and be 
> able to be even remotely sure that the kernel is going to work, and 
> that's been the case.  That combined with the rapid API changes, and 
> noone is developing a long lived stable kernel anymore means that 
> commercial support of this OS is being lost.  I'm in an odd situation 
> where because of NDAs and etc. I can not disclose any real details about 
> the commercial backers, but I'm sure they're not the only ones, and 
> probably much more important ones are getting frustrated.

In fact, I think you have chosen the wrong kernel base to start your
work. If you don't have the skills to follow large updates, you should
choose a stable kernel (stable code-wise). There is nothing ridiculous
in starting a project on 2.0, 2.2 or 2.4 right now. Given the past
experience of 2.4 and the time I can spend on kernel work, I would
not even consider basing anything on 2.6 before something like 2.6.20-25,
when it will hopefully settle down a bit. However, I have some 2.6 kernels
on very specific applications where I know I will not need to upgrade it,
eg: to make a bootloader using kexec, and on my home web server (hppa).

> Informational input can and often is as valuable as code.  Getting 
> someone to think of something they hadn't thought of can save that person 
> or the whole group lots of effors.
> 
> So, if you don't have anything USEFUL to retort back, shut up.  I'm 
> getting sick of hearing the people who can't contribute *ANYTHING NEW* to 
> the conversation and I'm in a very short mood.
> 
> The ... attitude and atmosphere here on LKML when someone brings up 
> something even slightly controversial is detrimental.  I know all of you 
> mean well.  But really.  If you're not contributing then you can stay 
> quiet just as well as the person you're complaining at.
> 
> This thread has been closed for what?   A week now?  I'm working on 
> trying to get the systems that are currently my big problems going, and 
> after that then I can focus more attention on the points I've brought up 
> earlier.  I'm only one person and just because I can't act immediately to 
> do something does not mean I won't.  Any of us who has an extremely busy 
> day job sure can understand that statement.

I understand what you feel but again, you did the wrong choice. You've
chosen 2.6 for various reasons, but you cannot expect its developers to
work the way that suits you best. Observe how kernels evolve and choose
another base to get less work. 2.4 can cost less than a few hours a month.
2.0 would need less than a few hours in a several years. Isn't it worth
spending more time porting your application to those kernels than you
currently spend trying to stay up to date with 2.6 ?

Willy

Re: Development tree, PLEASE?

[Sam Ravnborg]

On Thu, Feb 02, 2006 at 03:10:08PM -0500, Dave Jones wrote:
> In the current FC4 2.6.15.2 based update..
> 
> - 3 convenient 'make the buildsys life easier' patches
Something I should have a look at?
A pointer would be fine.

	Sam

Re: Development tree, PLEASE?

[Dave Jones]

On Thu, Feb 02, 2006 at 11:05:19PM +0100, Sam Ravnborg wrote:
 > On Thu, Feb 02, 2006 at 03:10:08PM -0500, Dave Jones wrote:
 > > In the current FC4 2.6.15.2 based update..
 > > 
 > > - 3 convenient 'make the buildsys life easier' patches
 > Something I should have a look at?
 > A pointer would be fine.

Nothing really amazing.. (I also miscounted -- 4 patches).

-rw-r--r--    1 davej    davej        4613 Dec 15 23:31 linux-2.6-build-nonintconfig.patch

Adds a 'nonintconfig' target that behaves like oldconfig, but doesn't
ask any questions (takes the default answer), and prints out a list
at the end of all the options it didn't know about.
(Handy when rebasing, as it means I get to add all the new options
 in one swoop).

-rw-r--r--    1 davej    davej         605 Dec 15 23:31 linux-2.6-build-reference-discarded-opd.patch

Think I posted this already, and it may even be in 16rc

-rw-r--r--    1 davej    davej        1686 Dec 15 23:31 linux-2.6-build-userspace-headers-warning.patch

adds a #error to includes if __KERNEL__ isn't being used
(We want people to use the headers from our glibc-kernheaders package,
 not from the kernel soucre).

-rw-r--r--    1 davej    davej        1753 Dec 15 23:31 linux-2.6-bzimage.patch

To get around some gynamstics in the rpm spec, defining a seperate build target
for every arch, we make every arch grok 'bzImage'. Fugly, but it keeps the
spec cleaner to maintain.

		Dave

Re: Development tree, PLEASE?

[David Weinehall]

On Thu, Feb 02, 2006 at 11:25:24AM -0700, Michael Loftis wrote:
> --On February 2, 2006 1:16:53 PM +0100 David Weinehall <tao@acc.umu.se> 
> wrote:
[snip]
> 
> The problem is that there's no more stable kernel first, and secondly that 
> there's not much if any pointers abotu the change.  People complained I 
> didn't have specific problems to solve, I don't CARE abotu any specific 
> problems.  I could hardly give a rats ass about devfs.  It's the whole new 
> development model that's the problem, and will only get worse for the 
> types of companies who I work with who normally right now support Linux 
> development.  I only brought up devfs because certain members of the 
> community can't see past bootstrings to the bigger issue.

If you don't give a rats ass about devfs (good!), don't bring it into
the discussion in the first place; it's not like that horse hasn't been
flogged to death already...

[snip]
> There's noone out there that does that, I'd LOVE to be able to pay for it 
> and not have to do it myself.

And there is a reason that there's noone out there that does it.
There's no community to do it, because it's too time consuming, with
little or no gain, and there's no commercial effort, because noone
has publicly announced, that "Hey, I'm willing to fork up EUR50000
per year to have (say) 2.6.14 maintained indefinitely as a stable
kernel branch with only security fixes, important bugfixes,
and minor device driver backports".  If you do, I bet someone will
come running.

[not biting the distro kernel bullet]

> Further, I'm not the only one.  I'm the only one who has enough asbestos 
> to jump onto LKML and say it because they all know that it's completely 
> hostile in here when someone brings up something that looks like a major 
> issue.

Ahhh.  The silent majority.  Yup, always a good backup to lean on.

> >o You think that 2.4.x isn't supporting enough new hardware,
> >  and yet you claim that adding new PCI ID:s is enough to add
> >  support for new hardware in most cases -- check
> 
> No I don't i said in MANY or atleast SOME.  further 2.4 is supposedly
> DEAD anyway.

For running any newer kind of hardware, yes, since the kernel needs
completely new framework for a lot of the new hardware.  And that's
not going to change even if you fork a stable kernel now.  A year from
now, that fork is going to be DEAD too; all development will continue
to go into the development kernel, since that's what all the distros
use (their users want it, to be able to use their new, cool hardware).

[snip]

> And it's only been the best because there's been a way for people to sue 

(I assume you mean use, not sue here?)

> it, get security patches even teh occasional new hardware support when 
> it's not disruptive.  Now that entire development model has changed into 
> something that noone uses because it doesn't' work for a project needing a 
> stable tree, such as a kernel.  It seems most people here just can't 
> understand how it can become a problem unfortunately because they can't 
> really see the big picture.  Everyone wants to take one little piece and 
> go hey hey see that's not a problem and then self satisfied at their 
> attacking and dismissal think they've solved the problem.  This is not a 
> single problem issue.

I think most people here understand that it can be a problem.  The thing
is, however, that since this is a project driven by voluntary developers,
not by commercial interest, there has to be a choice made somewhere:
do we focus on stability, risking to drive the developers away, or do
we focus on a development process which tries to avoid breaking things
in a too bad manner while keeping up the development pace as much as
possible?

The third option you seem to look for -- full stability AND active
development, which *sort of* worked during the old days of kernel
development, simply doesn't scale.  In fact, I'd say that the
development series we have now is a lot more is a lot more stable than
2.2.0 - 2.2.20, and 2.4.0 - 2.4.12 at least.  So what you might look
forward to if we go back to the old development model is one of these
options:

o People leaving the project because the development kernel is too
  broken (most of 2.1 series, part of 2.3-series, big chunks of
  2.5-series), and the stable series is too stale

o The new stable kernel, when branched, is too unstable, since noone
  tested it (because it was unstable)

o The development kernel, while quite unasable, newer ever becomes the
  stable kernel (which is pretty much the situation we have now)

[snip]

> I'm damned sick of the number of people who just *ATTACK* people who 
> contribute.  Constructive criticism is a form of contribution, feedback if 
> those words are too big for some to understand.  Because of the 

yes, *constructive* criticism is a form of contribution.  But your
posts in this thread so far hasn't been very constructive.

> development model changes there are projects not going to use Linux at 
> several companies that I work for contracting.  Because there is no way 
> that any single entity can look at 4+MB of compressed code changes and be 
> able to be even remotely sure that the kernel is going to work, and that's 
> been the case.

How about trying?  And, as mentioned earlier in the thread,
if a company is not prepared to spend money to either hire their own
kernel maintainers or pay someone else to do it, they have to play
by the rules of the community.  If you want to complain, do it by either
submitting bug reports / patches, or by writing checks.

The fact that this is free (and FREE) software means it doesn't cost
you anything to get the sources.  Noone has said that it does not cost
you anything to use it.  It does cost you, either in terms of compromise
(accepting the limitations that are in the kernel, accepting the fact
 that the development happens in different areas than you want it to),
or in terms of support deals.

The fact that free software comes to life to "scratch an itch" does
not mean that the developers necessarily want to scratch *your* itch...

> That combined with the rapid API changes, and noone is 
> developing a long lived stable kernel anymore means that commercial 
> support of this OS is being lost.  I'm in an odd situation where because 
> of NDAs and etc. I can not disclose any real details about the commercial 
> backers, but I'm sure they're not the only ones, and probably much more 
> important ones are getting frustrated.

Ahhh, again the silent majority.

Sure, I *bet* there are commercial companies that lose their interest
in the kernel for various reasons.  If you want a slower development
cycle, why not look at one of the *BSD's?

But if you have the impression that the tide goes away from the Linux
kernel, I'm sorry to disappoint you.  I work for Nokia's OSSO team.
We pushed out a products a few months ago based on the 2.6.1x (cannot
remember, sorry) kernel -- the Nokia 770 Internet Tablet, more and more
several NAS servers are surfacing that are Linux-based, multimedia
devices, cellphones (nothing from Nokia that I'm aware of, but other
companies have several models), etc.

> Informational input can and often is as valuable as code.  Getting someone 
> to think of something they hadn't thought of can save that person or the 
> whole group lots of effors.

If you *really* think that your "insight" was unique, start reading
a bit of backlog on LKML.  This discussion surfaces every few months,
and it always dies, since nobody *ever* comes up with the financial
backing for the effort, nor does anyone step up to do the long time
maintenance.  The 2.6.x.y kernel series did indeed come out of one
of these discussions, but that's it -- if the 6 months of stability
it will bring you isn't enough, pay someone to keep it alive longer.

> So, if you don't have anything USEFUL to retort back, shut up.  I'm 
> getting sick of hearing the people who can't contribute *ANYTHING NEW* to 
> the conversation and I'm in a very short mood.

Don't forget that you started the whole thing, without bring anything
new...  Read LKML, learn from past threads.

> The ... attitude and atmosphere here on LKML when someone brings up 
> something even slightly controversial is detrimental.  I know all of you 
> mean well.  But really.  If you're not contributing then you can stay 
> quiet just as well as the person you're complaining at.

Oh yes, the attitude and atmosphere here is raw and sometimes hostile.
One reason is that most of us get tired when the same useless
discussions resurface again and again and again.  It's one thing
if we get multiple bug reports, but discussions like this does not
really bring anything useful.  It's even less meaningful than the
usual flamewars we have here, and a lot less entertaining than
some of the ravings.

Another reason is that the posts like yours always start out
with something like "This is how things should be instead", or
"We must do this", or "The current foo sucks".

If your post instead had started with "From 2.6.15 and on I'm going to
maintain a stable kernel series which will only contain security fixes,
important bugfixes, and certain driver backports; at first I will do
this alone, but I hope to be able to hire 1-2 full time kernel hackers
to do this job.  Any takers?", I bet you'd have gotten a lot of good
response.

> This thread has been closed for what?   A week now?  I'm working on trying 
> to get the systems that are currently my big problems going, and after 
> that then I can focus more attention on the points I've brought up 
> earlier.  I'm only one person and just because I can't act immediately to 
> do something does not mean I won't.  Any of us who has an extremely busy 
> day job sure can understand that statement.

Yeah, because noone on this list than you has an extremely busy day job,
right?  I do spot your innuendo, but sorry, you're way off target.

I suspect that applies for most/all of the core kernel developers as
well (the ones that you're primarily aiming this at, since it's their
efforts you want to be reorganized).


Regards: David
-- 
 /) David Weinehall <tao@acc.umu.se> /) Northern lights wander      (\
//  Maintainer of the v2.0 kernel   //  Dance across the winter sky //
\)  http://www.acc.umu.se/~tao/    (/   Full colour fire           (/

Re: Development tree, PLEASE?

[Sam Ravnborg]

On Thu, Feb 02, 2006 at 05:10:25PM -0500, Dave Jones wrote:
> 
> -rw-r--r--    1 davej    davej        4613 Dec 15 23:31 linux-2.6-build-nonintconfig.patch
> 
> Adds a 'nonintconfig' target that behaves like oldconfig, but doesn't
> ask any questions (takes the default answer), and prints out a list
> at the end of all the options it didn't know about.
> (Handy when rebasing, as it means I get to add all the new options
>  in one swoop).
I have this around somewhere. hch did it but recall Roman did not
like it. It's in my pile of 'when I am in kconfig hacking mode' which
happens now and then.

> 
> -rw-r--r--    1 davej    davej         605 Dec 15 23:31 linux-2.6-build-reference-discarded-opd.patch
> 
> Think I posted this already, and it may even be in 16rc
Have applied some changes recently. Needs to come in via (or acked-by)
Keith Ownes though.

> -rw-r--r--    1 davej    davej        1686 Dec 15 23:31 linux-2.6-build-userspace-headers-warning.patch
> 
> adds a #error to includes if __KERNEL__ isn't being used
> (We want people to use the headers from our glibc-kernheaders package,
>  not from the kernel soucre).
Will not touch it. Combining 'kernel header files' and 'userspace' in 
same sentence generate far too much noise :-(

> 
> -rw-r--r--    1 davej    davej        1753 Dec 15 23:31 linux-2.6-bzimage.patch
> 
> To get around some gynamstics in the rpm spec, defining a seperate build target
> for every arch, we make every arch grok 'bzImage'. Fugly, but it keeps the
> spec cleaner to maintain.

Yup - seen it before. Did not like it.
Consistent use of KBUILD_IMAGE across relevant architectures should buy
you the same simplicity and a mergeable approach.

Thanks,
	Sam

Re: Development tree, PLEASE?

[Dave Jones]

On Thu, Feb 02, 2006 at 11:19:22PM +0100, Sam Ravnborg wrote:

 > > -rw-r--r--    1 davej    davej        1753 Dec 15 23:31 linux-2.6-bzimage.patch
 > > 
 > > To get around some gynamstics in the rpm spec, defining a seperate build target
 > > for every arch, we make every arch grok 'bzImage'. Fugly, but it keeps the
 > > spec cleaner to maintain.
 > 
 > Yup - seen it before. Did not like it.
 > Consistent use of KBUILD_IMAGE across relevant architectures should buy
 > you the same simplicity and a mergeable approach.

tbh, patches like this just sit there, as 'they just work', rarely need
rediffing, the benefit to other people of it being upstream are next to nil
(or someone else would've done it by now), and I'm more motivated to work on
real problems like finding out why we've 3 different flavours of slab corruption right now.

		Dave

Re: Development tree, PLEASE?

[Christopher Friesen]

Willy Tarreau wrote:

> Given the past
> experience of 2.4 and the time I can spend on kernel work, I would
> not even consider basing anything on 2.6 before something like 2.6.20-25,
> when it will hopefully settle down a bit.

Unfortunately there are many times when this simply isn't an option. 
I'm currently working on boards that simply are not supported on 2.4. 
Thus either we need to track 2.6, or else we need to pay someone to do 
it for us and hope they do a good job.

Of course what actually happens is that you pick a kernel version 
(hopefully you pick well) and then backport fixes.  Upgrading 
continuously simply isn't an option, as we have multiple vendors 
providing BSPs, drivers, patches, etc. and they're all supported only 
for that specific kernel version.  We can really only upversion the 
kernel once a year or so, if that often.

Chris

Re: Development tree, PLEASE?

[Sam Ravnborg]

On Thu, Feb 02, 2006 at 05:31:56PM -0500, Dave Jones wrote:
> On Thu, Feb 02, 2006 at 11:19:22PM +0100, Sam Ravnborg wrote:
> 
>  > > -rw-r--r--    1 davej    davej        1753 Dec 15 23:31 linux-2.6-bzimage.patch
>  > > 
>  > > To get around some gynamstics in the rpm spec, defining a seperate build target
>  > > for every arch, we make every arch grok 'bzImage'. Fugly, but it keeps the
>  > > spec cleaner to maintain.
>  > 
>  > Yup - seen it before. Did not like it.
>  > Consistent use of KBUILD_IMAGE across relevant architectures should buy
>  > you the same simplicity and a mergeable approach.
> 
> tbh, patches like this just sit there, as 'they just work', rarely need
> rediffing, the benefit to other people of it being upstream are next to nil
> (or someone else would've done it by now), and I'm more motivated to work on
> real problems like finding out why we've 3 different flavours of slab corruption right now.

Fair enough. I expected you to ask me to do it ;-)
Good luck with the slab stuff.

	Sam

Re: Development tree, PLEASE?

[Michael Loftis]

--On February 2, 2006 11:15:12 PM +0100 David Weinehall <tao@acc.umu.se> 
wrote:

>> And it's only been the best because there's been a way for people to sue
>
> (I assume you mean use, not sue here?)

Yeah, never trust a spelling  checker. :)

Re: Development tree, PLEASE?

[Roman Zippel]

Hi,

On Thu, 2 Feb 2006, Sam Ravnborg wrote:

> On Thu, Feb 02, 2006 at 05:10:25PM -0500, Dave Jones wrote:
> > 
> > -rw-r--r--    1 davej    davej        4613 Dec 15 23:31 linux-2.6-build-nonintconfig.patch
> > 
> > Adds a 'nonintconfig' target that behaves like oldconfig, but doesn't
> > ask any questions (takes the default answer), and prints out a list
> > at the end of all the options it didn't know about.
> > (Handy when rebasing, as it means I get to add all the new options
> >  in one swoop).
> I have this around somewhere. hch did it but recall Roman did not
> like it. It's in my pile of 'when I am in kconfig hacking mode' which
> happens now and then.

I'm not sure if I've even seen it, I usually don't poke around in rpms, so 
I can't even find it easily right now.

bye, Roman

Re: Development tree, PLEASE?

[Theodore Ts'o]

On Thu, Feb 02, 2006 at 11:19:22PM +0100, Sam Ravnborg wrote:
> On Thu, Feb 02, 2006 at 05:10:25PM -0500, Dave Jones wrote:
> > 
> > -rw-r--r--    1 davej    davej        4613 Dec 15 23:31 linux-2.6-build-nonintconfig.patch
> > 
> > Adds a 'nonintconfig' target that behaves like oldconfig, but doesn't
> > ask any questions (takes the default answer), and prints out a list
> > at the end of all the options it didn't know about.
> > (Handy when rebasing, as it means I get to add all the new options
> >  in one swoop).
> I have this around somewhere. hch did it but recall Roman did not
> like it. It's in my pile of 'when I am in kconfig hacking mode' which
> happens now and then.

It'd be nice if this one was included; we're using this in a set of
kernel patches which we deliver to a customer who needs a real-time
Linux solution, and we're using it for the same reason that
RHEL/Fedora is; it's really handy when rebasing to the latest kernel.

(Note to Michael Loftis re: the original thread: ++ to a list of "real
companies" that are using and tracking bleeding edge kernels for a
customer deliverable.  It _is_ doable if you aren't spending all of
your time whining....)

						- Ted

Re: Development tree, PLEASE?

[Willy Tarreau]

On Thu, Feb 02, 2006 at 04:31:58PM -0600, Christopher Friesen wrote:
> Willy Tarreau wrote:
> 
> >Given the past
> >experience of 2.4 and the time I can spend on kernel work, I would
> >not even consider basing anything on 2.6 before something like 2.6.20-25,
> >when it will hopefully settle down a bit.
> 
> Unfortunately there are many times when this simply isn't an option. 
> I'm currently working on boards that simply are not supported on 2.4. 
> Thus either we need to track 2.6, or else we need to pay someone to do 
> it for us and hope they do a good job.
> 
> Of course what actually happens is that you pick a kernel version 
> (hopefully you pick well) and then backport fixes.

This choice is valid when you have time or money for this. What I said
is people who have no skills, no time and no money should obviously
not consider 2.6 right now for a new project, it's moving too fast, and
spending 1 hour a week on it is not enough.

> Upgrading 
> continuously simply isn't an option, as we have multiple vendors 
> providing BSPs, drivers, patches, etc. and they're all supported only 
> for that specific kernel version.  We can really only upversion the 
> kernel once a year or so, if that often.

Agreed.

> Chris

Willy

Re: Development tree, PLEASE?

[Roman Zippel]

Hi,

On Thu, 2 Feb 2006, Dave Jones wrote:

> -rw-r--r--    1 davej    davej        4613 Dec 15 23:31 linux-2.6-build-nonintconfig.patch
> 
> Adds a 'nonintconfig' target that behaves like oldconfig, but doesn't
> ask any questions (takes the default answer), and prints out a list
> at the end of all the options it didn't know about.
> (Handy when rebasing, as it means I get to add all the new options
>  in one swoop).

You also get the default answers with 'yes "" | make oldconfig', but what 
exactly are you doing with the list of config options?
What are the changes to confdata.c for?

bye, Roman

Re: Development tree, PLEASE?

[Dave Jones]

On Fri, Feb 03, 2006 at 01:28:13PM +0100, Roman Zippel wrote:
 > Hi,
 > 
 > On Thu, 2 Feb 2006, Dave Jones wrote:
 > 
 > > -rw-r--r--    1 davej    davej        4613 Dec 15 23:31 linux-2.6-build-nonintconfig.patch
 > > 
 > > Adds a 'nonintconfig' target that behaves like oldconfig, but doesn't
 > > ask any questions (takes the default answer), and prints out a list
 > > at the end of all the options it didn't know about.
 > > (Handy when rebasing, as it means I get to add all the new options
 > >  in one swoop).
 > 
 > You also get the default answers with 'yes "" | make oldconfig', but what 
 > exactly are you doing with the list of config options?
 > What are the changes to confdata.c for?

Convenience more than anything.

It's to do with how the configs for Fedora/RHEL kernels are generated.
Rather than have a dozen separate .config files, and have to add a new
option to each of them, it's done in a 'templated' manner.

We have for eg, a config-generic, and then various config-$arch files,
which are munged together with a perl script to generate a final .config
that our build system eats for each arch it builds.

Having a list of all the new options together means that I can just cut-n-paste
that block of text into config-generic, and then drop out the ones that
should be per-arch.

I've felt it's another of those 'of little practical use to anyone not building
a Fedora/RHEL kernel' type patches that I've not bothered pushing it upstream.

		Dave

Re: Development tree, PLEASE?

[Dmitry Torokhov]

On 1/20/06, Marc Koschewski <marc@osknowledge.org> wrote:
>
> Let me repeat this with a clean 2.6.16-rc1 install without any patches and
> resync_timing of 5 tonite. I'll send the whole kern.log part from gdm login (the
> agpgart lines) until the mouse jumps then.
>

Marc,

I am still waiting on that debug log with jumpy mouse after resync.
Any chance you could send it to me, please?

--
Dmitry

Re: Development tree, PLEASE?

[Jan Engelhardt]

>I created one sometime ago.
>Different origin, same problem.
>
>https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=168045
>
Err.
This one is better:

https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=174190


Jan Engelhardt
-- 

Re: Development tree, PLEASE?

[Jan Engelhardt]

>> On Thu, 2 Feb 2006 17:10:25 -0500, Dave Jones wrote:
>> 
>> > -rw-r--r--    1 davej    davej        1686 Dec 15 23:31 linux-2.6-build-userspace-headers-warning.patch
>> > 
>> > adds a #error to includes if __KERNEL__ isn't being used
>> > (We want people to use the headers from our glibc-kernheaders package,
>> > not from the kernel soucre).
>> 
>>  Red Hat's headers are way out of date.
>> 
>>  Just try compiling this using FC4's latest kernheaders:
>> 
>>         ptrace(PTRACE_SETOPTIONS, child, NULL, PTRACE_O_TRACEFORK);
>> 
>>  PTRACE_O_TRACEFORK is undefined.
>
>
>what is the bugzilla number for this?
>

I created one sometime ago.
Different origin, same problem.

https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=168045



Jan Engelhardt
-- 

Re: Development tree, PLEASE?

[Arjan van de Ven]

On Tue, 2006-03-14 at 08:57 -0500, Chuck Ebbert wrote:
> In-Reply-To: <20060202221023.GJ11831@redhat.com>
> 
> On Thu, 2 Feb 2006 17:10:25 -0500, Dave Jones wrote:
> 
> > -rw-r--r--    1 davej    davej        1686 Dec 15 23:31 linux-2.6-build-userspace-headers-warning.patch
> > 
> > adds a #error to includes if __KERNEL__ isn't being used
> > (We want people to use the headers from our glibc-kernheaders package,
> > not from the kernel soucre).
> 
>  Red Hat's headers are way out of date.
> 
>  Just try compiling this using FC4's latest kernheaders:
> 
>         ptrace(PTRACE_SETOPTIONS, child, NULL, PTRACE_O_TRACEFORK);
> 
>  PTRACE_O_TRACEFORK is undefined.


what is the bugzilla number for this?

Re: Development tree, PLEASE?

[Chuck Ebbert]

In-Reply-To: <20060202221023.GJ11831@redhat.com>

On Thu, 2 Feb 2006 17:10:25 -0500, Dave Jones wrote:

> -rw-r--r--    1 davej    davej        1686 Dec 15 23:31 linux-2.6-build-userspace-headers-warning.patch
> 
> adds a #error to includes if __KERNEL__ isn't being used
> (We want people to use the headers from our glibc-kernheaders package,
> not from the kernel soucre).

 Red Hat's headers are way out of date.

 Just try compiling this using FC4's latest kernheaders:

        ptrace(PTRACE_SETOPTIONS, child, NULL, PTRACE_O_TRACEFORK);

 PTRACE_O_TRACEFORK is undefined.

 No wonder people use kernel headers despite being told not to.

-- 
Chuck
"Penguins don't come from next door, they come from the Antarctic!"

Re: Development tree, PLEASE?

[Michael Loftis]

--On January 21, 2006 1:38:46 AM +0000 Alan Cox <alan@lxorguk.ukuu.org.uk> 
wrote:

> On Gwe, 2006-01-20 at 13:56 -0700, Michael Loftis wrote:
>> and is fine once getty gets ahold of it, it's just during the initial
>> bootup phases where it's being used as the console either by the rc
>> scripts  or by the kernel that seems to go wonky.  It goes out during
>> the initial
>
> A bug where the serial console could get stuck on SMP IFF a kernel and a
> non kernel message were output at the same time did get fixed
> (yesterday) other than that I'm not aware of any problems in this area
> but the maintainer may have more ideas.
>
> Diff is tiny if you want to see if that is what you hit, although it
> would be remarkable co-incidence and luck if it was ;)

Coincidence I'm full of, and (bad) luck this week as well it seems.  I want 
to know who's black cat has been crossing my path.  This gives me a better 
direction to test it in.  The machines I have the problem with are all 
running SMP preemptible 2.6.8 on an HT machine with a single physical core, 
I'll try putting or booting them into a non-SMP kernel...if it's suddenly 
good, well....we found our rat.  That would though explain it pretty well 
since thinking about it, it doesn't happen in the debian installer nor... i 
think it's one of the gentoo installers or something...and those are 386 
non-SMP kernels.

Might've found some sort of wacky edge-case that can reproduce that bug 
reliably.  I'd be much appreciative if you pass a link or the diff itself 
along to me (or a specific bit to look for in archives/etc).  It might, or 
might not, be my little gremlin.  In the meantime I just leave off 
console=ttyS1,38400 and hold my breath while they boot.

>
>> printk output, or sometimes later...exactly when seems to be a bit of a
>> random thing.  Also it either causes, or is inputting NULL's or some
>> other  (consistent) garbage (CRLF? instead of CR?) on these same blades.
>> So you
>
> Never seen CR, nul reported. Would the blades happen to use rlogin to
> manage this remote serial do you know ?

No...telnet...though...I just realised I haven't verified that on anything 
but the BSD based telnet program on Mac OS X, and FreeBSD 4.11(ish), so 
really, it might be something there too, but again, 2.4 never sees these 
issues, and I'm really not sure what's getting into the stream, I think nul 
because I get a '^@' translated back at me, which IIRC is the 
representation for nul but lord knows if that's from the telnet client 
after it echos or what, I haven't done a packet dump of this gremlin, yet.

>
>> I think I have more kernel bugs and can go on, but I'll just be told
>> 'upgrade to 2.6.15' which is not an option in many cases if these are
>> indeed development releases, if only 'politically', but there are often
>> real costs involved.  And with nowhere to put patches that end up in
>
> Its hard to maintain an old release and just merge all the fixes into it
> backporting when neccessary. At the kernel summit before 2.6 this was
> discussed a lot. There are a small number of groups of people who wanted
> this for the long term. Said groups either maintain such trees and sell
> support/services for money, or rebuild the output of the former as a
> community project.
>
> It therefore seemed reasonable that those who want it should bear the
> cost, or figure out how to maintain such backports between themselves.

OK atleast I'm not total net.kook here.

>> maintenance releases we're forced to maintain our own private forks, and
>> likely, because of the GPL, also publish these forks and incur all the
>> costs associated with that directly, and hope they don't become
>> popular/wanted outside of the customer base they're intended for, or
>> skirt  the GPL, and only allow customers access to this stuff.
>
> The GPL is very careful about this. If you ship the sources to your
> customers then you have done your duty. If your customers choose to give
> it to others so be it. If the others ask you for changes then I believe
> the phrase is "business opportunity".
>
>> whatever their version numbers are.  I'm in an odd position of working
>> for  a web hosting company, *and* doing my own Linux consulting as well,
>> and  maintaining some 'embedded distros' used in these specific niche
>> applications.
>
> Embedded can be more problematic because it is harder to spread the
> load, but there are communities of people who looked at things like Red
> Hat Enterprise Linux and decided that they could use the code but didn't
> currently need/want the training, support and services that are what
> really makes it. One obvious example is Centos which is a community tree
> derived from the RHEL work, rebuilt, rebranded without
> support/services/etc and downloadable for free.

Yeah, embedded certainly is its own special little corner of heaven or hell 
depending on your view, or whims.