Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[James Clark]

Thank you for this (and the few other) sensible appraisal of my 'proposal'. 

I'm very surprised by the number of posts that have ranted about Open/Close 
source, GPL/taint issues etc. This is not about source code it is about 
making Linux usable by the masses. It may be technically superior to follow 
the current model, but if the barrier to entry is very high (and it is!) then 
the project will continue to be a niche project. A binary model doesn't alter 
the community or the benefits of public source code. I agree that it is an 
extra interface and will carry a performance hit - I think this is worth it. 
Windows has many faults but drivers are often compatible across major 
releases and VERY compatible across minor releases. It is no accident that it 
has 90% of the desktop market. If we are going to improve this situation this 
issue MUST be confronted.

I am currently collating the arguments for and (mostly) against the idea. If I 
don't get flamed in the meantime I may come back with more...

James



On Thursday 04 Sep 2003 2:02 am, you wrote:
> > Following my initial post yesterday please find attached my
> > proposal for a binary 'plugin' interface:
>
> There was one, called UDI. It was rejected.
>
> > I believe that sometimes the ultimate goals of stability and
> > portability get lost in the debate on OSS and desire to allow
> > anyone to  contribute.
>
> If "stability" means "no changes", then don't upgrade
> for a few years.
>
> If "stability" means "no crashes", then a binary
> interface is harmful. Extra layers mean extra bugs,
> and the layers make debugging more difficult.
>
> Portability??? Linux beats every other OS for that.
>
> > 3. Design 'plugin' interface to be extensible as possible
> > and then rarely remove support from interface. Extending
> > interface is fine but should be done in a considered way
> > to avoid interface bloat. Suggest interface supports
> > dependant 'plugins'
>
> Removal of crud is important. We keep things simple,
> fast, and small.
>
> > 4. Allow 'plugins' to be bypassed at boot - perhaps a
> > minimal 'known good' list
>
> This works already. Put "init=/bin/bash" on the kernel
> command line (at the lilo, grub, or syslinux prompt) and
> you will bypass all of the start-up scripts. That will
> cause you to skip any modules (drivers, plugins, whatever)
> that would be loaded by your boot scripts.
>
> > 5. Ultimately, even FS 'plugins' could be created
> > although IPL would be required to load these.
>
> IPL is some old IBM mainframe term. In the Linux and
> UNIX world we call it boot, bootstrap, start-up, etc.
>
> We can already make our filesystem a loadable module.
> Your boot loader (lilo, grub, or syslinux) has the
> ability to load an "initrd". This gives you something
> kind of like a ramdisk, but without the wasted space.
> The scripts on the initrd can then load drivers for
> your disk (IDE, SCSI, etc.) and filesystem (ext2, ext3,
> reiserfs, reiser4, jfs, xfs, etc.). After the drivers
> are loaded, there's a mount() syscall, a pivit_root()
> syscall... and soon enough the initrd is gone and you
> have the real root filesystem mounted.
>
> > 1. Make Linux easier to use
>
> I doubt it. There would be a promise of binary
> compatibility that wouldn't be possible to maintain.
> Microsoft fails often enough; notice how Win98
> broke SCSI drivers that people were using.
>
> Without such a promise, people don't grow to depend
> on an interface that ultimately can't be supported.
>
> > 2. The ability to replace even very core Kernel
> > components without a restart.
>
> That's a fantasy, more-or-less. Binary patches can
> be done on a case-by-case basis at great expense.
> Novell did it sometimes. Microsoft didn't even try.
>
> > 3. Allow faulty 'plugins' to be fixed/replaced in
> > isolation. No other system impact.
>
> Linux supports this as well as can be. Any driver
> (module, plugin, whatever) has the ability to crash
> your system BY DEFINITION of having the necessary
> power to control your hardware.
>
> > 4. 'Plugins' could create their own interfaces as
> > load time. This would remove the need to pre-populate /dev.
>
> We have this as an option; we call it devfs.
> It has a problem: permission changes will not
> persist across a reboot.
>
> > 5. Remove need for joe soap user to often recompile Kernel.
>
> Already done: buy a CD-ROM from Red Hat
>
> > 6. Remove link between specific module versions
> > and Kernel versions.
>
> It's partly done (optional), and partly not desireable.
>
> > 7. Reduce need for major Kernel releases. Allow effort
> > to concentrate on improving Kernel not maintaining
> > ever increasing Kernel source that includes support
> > for the 'Kitchen Sink'
>
> No. It's damn nice to have all the source code in one
> place, all updated together, supporting whatever old
> hardware one may have.
>
> It's only recently (2.5.xx kernel series) that Linux
> dropped support for the PC-XT hard drive interface.
> Know what that is? It was introduced around 1981 maybe.
> Support was only dropped because all the disks have died.
>
> > 8. Make core Kernel more stable. Less releases and
> > less changes mean less bugs. It would be easy to
> > identify offending 'plugin' by simply starting up
> > the Kernel with it disabled.
>
> We can do this now. It's not a kernel issue though.
> Starting up with things disabled is a feature of
> your boot scripts, which are provided by your Linux
> distribution. See your Red Hat documentation, or
> your SuSE documentation, etc.
>
> > 9. Remove need for modules to be maintained in
> > sync with each Kernel thus  freeing 'module' developers
> > to add improved features or work on new projects.
>
> Then nobody would maintain them.

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Mike Fedyk]

On Thu, Sep 04, 2003 at 09:14:45PM +0100, James Clark wrote:
> Thank you for this (and the few other) sensible appraisal of my 'proposal'. 
> 
> I'm very surprised by the number of posts that have ranted about Open/Close 
> source, GPL/taint issues etc. This is not about source code it is about 
> making Linux usable by the masses. It may be technically superior to follow 
> the current model, but if the barrier to entry is very high (and it is!) then 
> the project will continue to be a niche project. A binary model doesn't alter 
> the community or the benefits of public source code. I agree that it is an 
> extra interface and will carry a performance hit - I think this is worth it. 

The thing is, most Linux developers (and I'm sure it's above 51% or maybe
they're just louder?) want drivers to be GPL compatible open source.  Having
a static binary driver interface just doesn't mix very well for that.  And
as things happen (and how it should be), in a well kept stable series, the
binary interfaces won't change that much.  But it will change for different
options, like SMP, preempt, numa, etc.

> Windows has many faults but drivers are often compatible across major 
> releases and VERY compatible across minor releases. It is no accident that it 
> has 90% of the desktop market. If we are going to improve this situation this 
> issue MUST be confronted.

Have you ever seen the source code available for a windows driver?  Windows
doesn't let you customize the kernel.  You just get what they give you.
With the customization possible in Linux you get many advantages, and the
disadvantage that the binary interface can change depending on the compile
options.

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[James Clark]

I agree that at first sight the two concepts (Binary 'plugins' and GPL) don't 
mix well but this is actually FUD which obscures the issue of making the 
kernel much easier to deal with for the masses. Like it or not, 99+% of 
'potential users' don't want/need to recompile their whole kernel, with the 
risks that it has, to add one minor feature.

James


On Thursday 04 Sep 2003 9:27 pm, Mike Fedyk wrote:
> On Thu, Sep 04, 2003 at 09:14:45PM +0100, James Clark wrote:
> > Thank you for this (and the few other) sensible appraisal of my
> > 'proposal'.
> >
> > I'm very surprised by the number of posts that have ranted about
> > Open/Close source, GPL/taint issues etc. This is not about source code it
> > is about making Linux usable by the masses. It may be technically
> > superior to follow the current model, but if the barrier to entry is very
> > high (and it is!) then the project will continue to be a niche project. A
> > binary model doesn't alter the community or the benefits of public source
> > code. I agree that it is an extra interface and will carry a performance
> > hit - I think this is worth it.
>
> The thing is, most Linux developers (and I'm sure it's above 51% or maybe
> they're just louder?) want drivers to be GPL compatible open source. 
> Having a static binary driver interface just doesn't mix very well for
> that.  And as things happen (and how it should be), in a well kept stable
> series, the binary interfaces won't change that much.  But it will change
> for different options, like SMP, preempt, numa, etc.
>
> > Windows has many faults but drivers are often compatible across major
> > releases and VERY compatible across minor releases. It is no accident
> > that it has 90% of the desktop market. If we are going to improve this
> > situation this issue MUST be confronted.
>
> Have you ever seen the source code available for a windows driver?  Windows
> doesn't let you customize the kernel.  You just get what they give you.
> With the customization possible in Linux you get many advantages, and the
> disadvantage that the binary interface can change depending on the compile
> options.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Mike Fedyk]

On Thu, Sep 04, 2003 at 10:16:03PM +0100, James Clark wrote:
> I agree that at first sight the two concepts (Binary 'plugins' and GPL) don't 
> mix well but this is actually FUD which obscures the issue of making the 
> kernel much easier to deal with for the masses. 

There is some point in easing the use of external modules with a binary
interface.  Case in point, winmodems & nvidia.  But the question is if we
really want to encourage binary-only modules, which most in the community
don't.

In Linux, drivers are kernel modules, and that has the ability to completly
hose the rest of the kernel if there are bugs in the module.  You won't get
around this with a static binary interface unless you put a lot of overhead
and layering between a module and the kernel.  Right now, once a module is
loaded, it IS now part of the kernel.

> Like it or not, 99+% of 
> 'potential users' don't want/need to recompile their whole kernel, with the
> risks that it has, to add one minor feature.

And they don't.  If they don't know how, or don't want to, they should be
using a distribution kernel that does it for them.

Are you saying that you want to add or change something now that is not
modularized yet?

If so what and how?  Yes you can say with this nice binary interface you can
add whatever feature you want with a nice new driver.  Well, you can do that
now.  All you have to do is compile against the source of the kernel you're
running, and you can insert it into the kernel without rebooting.

And if you're trying to change something in a running kernel that isn't a
module you can unload (so the root filesystem is out at the very least),
then you're talking about an entirely new can of worms.

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[insecure]

On Friday 05 September 2003 00:16, James Clark wrote:
> I agree that at first sight the two concepts (Binary 'plugins' and GPL)
> don't mix well but this is actually FUD which obscures the issue of making
> the kernel much easier to deal with for the masses. Like it or not, 99+% of
> 'potential users' don't want/need to recompile their whole kernel, with the
> risks that it has, to add one minor feature.

Those 99+% are going to use a distro. A popular Linux distro
is a rough equivalent of a Windows install disk in terms of
ease of use, feature set and intended audience.
Distro can handle this problem (of driver ABI or absence thereof)
in any way it wants. I can imagine it can even go to the great pains
and introduce binary driver ABI. But unlikely.

OTOH I believe core kernel folks will not accept your proposal.
Stay a bit longer on lkml and you will understand why.
--
vda

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[jdow]

From: "Mike Fedyk" <mfedyk@matchmail.com>

> Have you ever seen the source code available for a windows driver?
Windows
> doesn't let you customize the kernel.  You just get what they give you.
> With the customization possible in Linux you get many advantages, and the
> disadvantage that the binary interface can change depending on the compile
> options.

As a matter of fact, yes. Their (expensive) developer kits have SOME
examples. Of course, much of the information you need for any sort of
driver that is just a weensie bit strange is completely missing. It
makes a specialized USB serial port to MIDI driver very difficult to
build and even worse to deploy neatly without a lot of user interaction.

{^_^}

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Rik van Riel]

On Thu, 4 Sep 2003, James Clark wrote:

> I'm very surprised by the number of posts that have ranted about
> Open/Close source, GPL/taint issues etc. This is not about source code
> it is about making Linux usable by the masses.

How would "making it easier to include impossible to debug
device drivers" help towards your goal of making Linux more
usable ?

-- 
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[James Clark]

Why would binary drivers be any harder to debug than the existing binary 
kernel. If you want to debug something use the source code. My proposal 
doesn't remove the need for quality public source code but it does isolate 
the kernel components and allow for 'plugin' use on different kernels both 
old and new.

If a relatively small kernel component can be turned on/off and upgraded at 
will, without changing ANYTHING else, this would be a big step forward.

James


On Thursday 04 Sep 2003 9:29 pm, you wrote:
> On Thu, 4 Sep 2003, James Clark wrote:
> > I'm very surprised by the number of posts that have ranted about
> > Open/Close source, GPL/taint issues etc. This is not about source code
> > it is about making Linux usable by the masses.
>
> How would "making it easier to include impossible to debug
> device drivers" help towards your goal of making Linux more
> usable ?

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Alan Cox]

On Iau, 2003-09-04 at 22:12, James Clark wrote:
> If a relatively small kernel component can be turned on/off and upgraded at 
> will, without changing ANYTHING else, this would be a big step forward.

It already can, the install tool just happens to involve invoking gcc
and wrapping it nicely. People have already solved that. I'm actually
less worried in some ways about the binary only module wrapper question
because source wrappers around a binary module to handle glueing them
together puts the shipper in conflict with a patent filing owned by a
very large hardware company as well as in GPL questionmarks..

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Bartlomiej Zolnierkiewicz]

Heh.  You want stable plugin API.

It will always end as bloat:

(a) you add plugin API
(b) its not sufficient, so you add revision 1 of the API
    (you still have to support rev 0 because some plugins are binary-only)
(c) its not sufficient, so you add revision 2 of the API
    (now you have 3 versions of API to maintain)
...

Alternatively you can start with over-designed API from the beginning :-).

--bartlomiej

On Thursday 04 of September 2003 23:12, James Clark wrote:
> Why would binary drivers be any harder to debug than the existing binary
> kernel. If you want to debug something use the source code. My proposal
> doesn't remove the need for quality public source code but it does isolate
> the kernel components and allow for 'plugin' use on different kernels both
> old and new.
>
> If a relatively small kernel component can be turned on/off and upgraded at
> will, without changing ANYTHING else, this would be a big step forward.
>
> James
>
> On Thursday 04 Sep 2003 9:29 pm, you wrote:
> > On Thu, 4 Sep 2003, James Clark wrote:
> > > I'm very surprised by the number of posts that have ranted about
> > > Open/Close source, GPL/taint issues etc. This is not about source code
> > > it is about making Linux usable by the masses.
> >
> > How would "making it easier to include impossible to debug
> > device drivers" help towards your goal of making Linux more
> > usable ?

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Jamie Lokier]

James Clark wrote:
> Why would binary drivers be any harder to debug than the existing binary 
> kernel. If you want to debug something use the source code. My proposal 
> doesn't remove the need for quality public source code but it does isolate 
> the kernel components and allow for 'plugin' use on different kernels both 
> old and new.

Your first statement doesn't make sense.

We don't debug binary kernels by themselves - we debug them with source.

Binary drivers that come without source are almost impossible to debug.

If we can't debug the drivers, we'll soon have a Linux community
that's full of broken drivers.

If broken binary-only drivers becomes significant for many users,
Linux will become much less fun - and it may harm its adoption and
improvment.

Some would say the problem of binary-only drivers is already
significantly harming the development process: think of the people
running 2.4-only drivers who cannot be involved in 2.6 testing or
development.

Also the rate at which new open source drivers are produced will
decrease, because there will be less useful source for authors to
learn from.

-- Jamie

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Richard B. Johnson]

On Thu, 4 Sep 2003, James Clark wrote:

> Thank you for this (and the few other) sensible appraisal of my 'proposal'.
>
> I'm very surprised by the number of posts that have ranted about Open/Close
> source, GPL/taint issues etc. This is not about source code it is about
> making Linux usable by the masses. It may be technically superior to follow
> the current model, but if the barrier to entry is very high (and it is!) then
> the project will continue to be a niche project. A binary model doesn't alter
> the community or the benefits of public source code. I agree that it is an
> extra interface and will carry a performance hit - I think this is worth it.
> Windows has many faults but drivers are often compatible across major
> releases and VERY compatible across minor releases. It is no accident that it
> has 90% of the desktop market. If we are going to improve this situation this
> issue MUST be confronted.
>
> I am currently collating the arguments for and (mostly) against the idea. If I
> don't get flamed in the meantime I may come back with more...
>
> James
>


The problem with Windows users is not that they are
stupid or even uneducated. It's just that they have been
taught that using a widely-used operating system that
is defective in design and implementation is the way
that operating systems should be.

They have been taught that bugs are normal. "Every program
has bugs. It is impossible to check all possible execution
paths to verify that there are no bugs, etc." This kind
of teaching comes about when the teachers know little or
nothing about their subjects and simply parrot what they
have read in literature that, in many cases, has been written
by the very persons who are incapable of writing bug-free code.

They have been taught that the secret inner workings of
the kernel are best treated like the objects of an object-
oriented design. In fact, they are taught that it's
harmful for a programmer, much less a user, to understand
the underlying workings of the operating system. This
is taught because Microsoft doesn't want you to know what
an abysmal abortion the operating system is.

Before Microsoft, nobody would dare sell a product that
contained no warranty at all. Somehow, Microsoft has taught
its customers that they should never expect software to
actually work. Somehow, their lawyers have replaced the
usual; "We warrant this product to be free of defects in
workmanship and design for a period ..."  to a disclaimer
that many persons think is some kind of a warranty. In
most cases, you can't even take a defective CD, defective
because it can't be read, back to the vendor without
encountering; "On it's software. Nobody warrants that!"

You can thank Microsoft for this.

There have been billions and billions of dollars of
lost productivity in industry because of this defective
Operating System.  In one company alone, there was
over one million dollars lost last year to defective
software. You can look at the financial filings of
many of the publically-traded companies and see write-
offs due to defective software in this order of magnitude.
Multiply that by the number of companies in the world to
get the big picture.

And, somehow, companies still keep using that garbage!

Linux is something different. It strives for perfection.
There is no way in hell that you are going to add some
Microsoft-compatible driver interface to Linux. There
are no drivers that could ever work on Windows and
somehow work on Linux. These two systems are mutually
exclusive, Alpha-Omega, 0->inf, good-evil, absolutely the
antithesis of each other. I hear that's why Microsoft is
attempting to kill Linux by funding the SCO lawsuit (check
this week's EETimes, I didn't make it up).

Now, you propose to introduce a driver interface that is
defective in concept. You propose this because you just
don't get it. You just don't know anything about Operating
Systems and you just don't know anything about Linux.

It may not be your fault. There are lots of people who
haven't a clue because they have been taught things that
are simply not true at all. And, once you repeat a falsehood
enough times it becomes accepted as fact.

Before you can become qualified to propose a different
driver interface, you need to learn about Operating Systems.
I know that you do not know anything about Operating Systems
in general, because of your proposal.

I suggest that you read a book like "Developing Your Own
32-bit Operating System", Burges, H.W.Sams, ISBN 0-627-30655-7
This is an interesting book because it is not about Windows
and it's not about Linux. It's about a roll-your-own Operating
System. It even comes with a CD and you can boot a home-grown
system on your PC.

Now, I don't care if you studied Operating Systems in College.
In fact, that just might be the reason why you don't know what
you are talking about. Most such college courses in this subject
are absolute crap, written by Masters candidates who learned
a bunch of words and coined a few of their own.

After you learn about Operating Systems, then you might be able
to add some new capability to Linux. You know, they accept patches
here. If you've got a better way, you make a patch and we'll all
try it. It's really that simple.

Cheers,
Dick Johnson
Penguin : Linux version 2.4.22 on an i686 machine (794.73 BogoMips).
            Note 96.31% of all statistics are fiction.

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[James Clark]

This is exactly the kind of FUD that is slowing Linux development and 
preventing wider use. Windows, for all its faults is a VERY BIG commercial 
success. You (and I) may believe that MS have persued some pretty bad 
practises to get where they are but the FACT is they are there and YOU are 
here.  You may (I don't) believe that Windows is inherently unstable. This is 
FUD. It mostly works, sometimes it doesn't, but in total the number of 
working hours of PRODUCTIVE use from it is many orders of magnitude greater. 
Multiple the number of Windows users in the world by their working time and 
then do the same for Linux! 

I have not proposed a driver model that is compatible with the Windows DDK. 
This is pure invention from the usual school of 'Windows v Linux, Linux is 
better because we made it'. The Linux driver model could be much better and 
hence the OS could escape the niche box it currently is in. Please ask Joe 
User how he feels about rebuilding his whole OS to add IP6 support to an 
existing stable system etc.

You have accused me of not knowing anything about OS design. I don't what your 
knowledge is, so I won't try and score points by telling you about mine - 
what I do know is that you obviously don't appreciate what is holding Linux 
back - You.

James



On Thursday 04 Sep 2003 10:29 pm, you wrote:
> On Thu, 4 Sep 2003, James Clark wrote:
> > Thank you for this (and the few other) sensible appraisal of my
> > 'proposal'.
> >
> > I'm very surprised by the number of posts that have ranted about
> > Open/Close source, GPL/taint issues etc. This is not about source code it
> > is about making Linux usable by the masses. It may be technically
> > superior to follow the current model, but if the barrier to entry is very
> > high (and it is!) then the project will continue to be a niche project. A
> > binary model doesn't alter the community or the benefits of public source
> > code. I agree that it is an extra interface and will carry a performance
> > hit - I think this is worth it. Windows has many faults but drivers are
> > often compatible across major releases and VERY compatible across minor
> > releases. It is no accident that it has 90% of the desktop market. If we
> > are going to improve this situation this issue MUST be confronted.
> >
> > I am currently collating the arguments for and (mostly) against the idea.
> > If I don't get flamed in the meantime I may come back with more...
> >
> > James
>
> The problem with Windows users is not that they are
> stupid or even uneducated. It's just that they have been
> taught that using a widely-used operating system that
> is defective in design and implementation is the way
> that operating systems should be.
>
> They have been taught that bugs are normal. "Every program
> has bugs. It is impossible to check all possible execution
> paths to verify that there are no bugs, etc." This kind
> of teaching comes about when the teachers know little or
> nothing about their subjects and simply parrot what they
> have read in literature that, in many cases, has been written
> by the very persons who are incapable of writing bug-free code.
>
> They have been taught that the secret inner workings of
> the kernel are best treated like the objects of an object-
> oriented design. In fact, they are taught that it's
> harmful for a programmer, much less a user, to understand
> the underlying workings of the operating system. This
> is taught because Microsoft doesn't want you to know what
> an abysmal abortion the operating system is.
>
> Before Microsoft, nobody would dare sell a product that
> contained no warranty at all. Somehow, Microsoft has taught
> its customers that they should never expect software to
> actually work. Somehow, their lawyers have replaced the
> usual; "We warrant this product to be free of defects in
> workmanship and design for a period ..."  to a disclaimer
> that many persons think is some kind of a warranty. In
> most cases, you can't even take a defective CD, defective
> because it can't be read, back to the vendor without
> encountering; "On it's software. Nobody warrants that!"
>
> You can thank Microsoft for this.
>
> There have been billions and billions of dollars of
> lost productivity in industry because of this defective
> Operating System.  In one company alone, there was
> over one million dollars lost last year to defective
> software. You can look at the financial filings of
> many of the publically-traded companies and see write-
> offs due to defective software in this order of magnitude.
> Multiply that by the number of companies in the world to
> get the big picture.
>
> And, somehow, companies still keep using that garbage!
>
> Linux is something different. It strives for perfection.
> There is no way in hell that you are going to add some
> Microsoft-compatible driver interface to Linux. There
> are no drivers that could ever work on Windows and
> somehow work on Linux. These two systems are mutually
> exclusive, Alpha-Omega, 0->inf, good-evil, absolutely the
> antithesis of each other. I hear that's why Microsoft is
> attempting to kill Linux by funding the SCO lawsuit (check
> this week's EETimes, I didn't make it up).
>
> Now, you propose to introduce a driver interface that is
> defective in concept. You propose this because you just
> don't get it. You just don't know anything about Operating
> Systems and you just don't know anything about Linux.
>
> It may not be your fault. There are lots of people who
> haven't a clue because they have been taught things that
> are simply not true at all. And, once you repeat a falsehood
> enough times it becomes accepted as fact.
>
> Before you can become qualified to propose a different
> driver interface, you need to learn about Operating Systems.
> I know that you do not know anything about Operating Systems
> in general, because of your proposal.
>
> I suggest that you read a book like "Developing Your Own
> 32-bit Operating System", Burges, H.W.Sams, ISBN 0-627-30655-7
> This is an interesting book because it is not about Windows
> and it's not about Linux. It's about a roll-your-own Operating
> System. It even comes with a CD and you can boot a home-grown
> system on your PC.
>
> Now, I don't care if you studied Operating Systems in College.
> In fact, that just might be the reason why you don't know what
> you are talking about. Most such college courses in this subject
> are absolute crap, written by Masters candidates who learned
> a bunch of words and coined a few of their own.
>
> After you learn about Operating Systems, then you might be able
> to add some new capability to Linux. You know, they accept patches
> here. If you've got a better way, you make a patch and we'll all
> try it. It's really that simple.
>
> Cheers,
> Dick Johnson
> Penguin : Linux version 2.4.22 on an i686 machine (794.73 BogoMips).
>             Note 96.31% of all statistics are fiction.

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Alan Cox]

On Iau, 2003-09-04 at 22:51, James Clark wrote:
> I have not proposed a driver model that is compatible with the Windows DDK. 
> This is pure invention from the usual school of 'Windows v Linux, Linux is 
> better because we made it'. The Linux driver model could be much better and 
> hence the OS could escape the niche box it currently is in. Please ask Joe 
> User how he feels about rebuilding his whole OS to add IP6 support to an 
> existing stable system etc.

Joe User got IPV6 from his vendor as a standard component, and if he
didn't the large number of app patches his old distro need outweigh the
kernel.

However for drivers its the same - its a _source_ level interface.

For example 

	spin_lock(&lock)

compiles to different things for uniprocessor/SMP kernels and the
difference is worth real speed. So your binary interface roughly
speaking depends on

-	core options selected
-	compiler (gcc2 v gcc3)
-	SMP v UP
-	Highmem v non highmem
-	CPU target type
-	4G/4G split in 2.6 case

and a few more, while your source interface is pretty stable.

So what does that mean for someone adding a module

1.	It needs compiling
2.	If you want trivial end user ease then you need to wrap the
compilation up as part of the stuff the user never sees.

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Martin Mares]

Hello!

> It mostly works, sometimes it doesn't, but in total the number of 
> working hours of PRODUCTIVE use from it is many orders of magnitude greater. 

Maybe the other people working on Linux feel different, but I just don't
care about the masses and about anybody's commercial success. If they like
using Windows, well, they deserve it. I'm helping to create a system _I_
and the people around me like.

About drivers: People around me who are using Windows do have many
problems (want to use three different USB scanners simultaneously? eh,
forget about that), people using Linux have slightly less or maybe the
same amount of problems, but guess what -- for the Linux users there
usually is somebody around who knows how to fix the things while Windows
users are helplessly reinstalling their binary-only stuff nobody really
understands. That's a real story and I guess it's a very typical one.
And it's the reason why almost nobody here likes binary-only drivers.

> Multiple the number of Windows users in the world by their working time and 
> then do the same for Linux! 

Well, you can multiply the number of insects in this universe by their
working time and get an even larger number, but does that mean anything
except you are skilled in arithmetic?

				Have a nice fortnight
-- 
Martin `MJ' Mares   <mj@ucw.cz>   http://atrey.karlin.mff.cuni.cz/~mj/
Faculty of Math and Physics, Charles University, Prague, Czech Rep., Earth
First law of socio-genetics: Celibacy is not hereditary.

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Gustav Petersson]

James Clark wrote:

>This is exactly the kind of FUD that is slowing Linux development and 
>preventing wider use. Windows, for all its faults is a VERY BIG commercial 
>success. You (and I) may believe that MS have persued some pretty bad 
>practises to get where they are but the FACT is they are there and YOU are 
>here.  You may (I don't) believe that Windows is inherently unstable. This is 
>FUD. It mostly works, sometimes it doesn't, but in total the number of 
>working hours of PRODUCTIVE use from it is many orders of magnitude greater. 
>Multiple the number of Windows users in the world by their working time and 
>then do the same for Linux! 
>
>I have not proposed a driver model that is compatible with the Windows DDK. 
>This is pure invention from the usual school of 'Windows v Linux, Linux is 
>better because we made it'. The Linux driver model could be much better and 
>hence the OS could escape the niche box it currently is in. Please ask Joe 
>User how he feels about rebuilding his whole OS to add IP6 support to an 
>existing stable system etc.
>  
>
If i didn't know better I would say you haven't even tried linux. Joe 
User doesn't have to recompile anything. If the distribution he is using 
is any good (and most are) there is no need to recompile anything, it's 
already there as a module.

>You have accused me of not knowing anything about OS design. I don't what your 
>knowledge is, so I won't try and score points by telling you about mine - 
>what I do know is that you obviously don't appreciate what is holding Linux 
>back - You.
>
>James
>
>  
>
Most kernel developers don't get paid for what they do, they do it for 
fun and prestige. Reaching "the masses" isn't necessary the goal.

>
>On Thursday 04 Sep 2003 10:29 pm, you wrote:
>  
>
>>On Thu, 4 Sep 2003, James Clark wrote:
>>    
>>
>>>Thank you for this (and the few other) sensible appraisal of my
>>>'proposal'.
>>>
>>>I'm very surprised by the number of posts that have ranted about
>>>Open/Close source, GPL/taint issues etc. This is not about source code it
>>>is about making Linux usable by the masses. It may be technically
>>>superior to follow the current model, but if the barrier to entry is very
>>>high (and it is!) then the project will continue to be a niche project. A
>>>binary model doesn't alter the community or the benefits of public source
>>>code. I agree that it is an extra interface and will carry a performance
>>>hit - I think this is worth it. Windows has many faults but drivers are
>>>often compatible across major releases and VERY compatible across minor
>>>releases. It is no accident that it has 90% of the desktop market. If we
>>>are going to improve this situation this issue MUST be confronted.
>>>
>>>I am currently collating the arguments for and (mostly) against the idea.
>>>If I don't get flamed in the meantime I may come back with more...
>>>
>>>James
>>>      
>>>
>>The problem with Windows users is not that they are
>>stupid or even uneducated. It's just that they have been
>>taught that using a widely-used operating system that
>>is defective in design and implementation is the way
>>that operating systems should be.
>>
>>They have been taught that bugs are normal. "Every program
>>has bugs. It is impossible to check all possible execution
>>paths to verify that there are no bugs, etc." This kind
>>of teaching comes about when the teachers know little or
>>nothing about their subjects and simply parrot what they
>>have read in literature that, in many cases, has been written
>>by the very persons who are incapable of writing bug-free code.
>>
>>They have been taught that the secret inner workings of
>>the kernel are best treated like the objects of an object-
>>oriented design. In fact, they are taught that it's
>>harmful for a programmer, much less a user, to understand
>>the underlying workings of the operating system. This
>>is taught because Microsoft doesn't want you to know what
>>an abysmal abortion the operating system is.
>>
>>Before Microsoft, nobody would dare sell a product that
>>contained no warranty at all. Somehow, Microsoft has taught
>>its customers that they should never expect software to
>>actually work. Somehow, their lawyers have replaced the
>>usual; "We warrant this product to be free of defects in
>>workmanship and design for a period ..."  to a disclaimer
>>that many persons think is some kind of a warranty. In
>>most cases, you can't even take a defective CD, defective
>>because it can't be read, back to the vendor without
>>encountering; "On it's software. Nobody warrants that!"
>>
>>You can thank Microsoft for this.
>>
>>There have been billions and billions of dollars of
>>lost productivity in industry because of this defective
>>Operating System.  In one company alone, there was
>>over one million dollars lost last year to defective
>>software. You can look at the financial filings of
>>many of the publically-traded companies and see write-
>>offs due to defective software in this order of magnitude.
>>Multiply that by the number of companies in the world to
>>get the big picture.
>>
>>And, somehow, companies still keep using that garbage!
>>
>>Linux is something different. It strives for perfection.
>>There is no way in hell that you are going to add some
>>Microsoft-compatible driver interface to Linux. There
>>are no drivers that could ever work on Windows and
>>somehow work on Linux. These two systems are mutually
>>exclusive, Alpha-Omega, 0->inf, good-evil, absolutely the
>>antithesis of each other. I hear that's why Microsoft is
>>attempting to kill Linux by funding the SCO lawsuit (check
>>this week's EETimes, I didn't make it up).
>>
>>Now, you propose to introduce a driver interface that is
>>defective in concept. You propose this because you just
>>don't get it. You just don't know anything about Operating
>>Systems and you just don't know anything about Linux.
>>
>>It may not be your fault. There are lots of people who
>>haven't a clue because they have been taught things that
>>are simply not true at all. And, once you repeat a falsehood
>>enough times it becomes accepted as fact.
>>
>>Before you can become qualified to propose a different
>>driver interface, you need to learn about Operating Systems.
>>I know that you do not know anything about Operating Systems
>>in general, because of your proposal.
>>
>>I suggest that you read a book like "Developing Your Own
>>32-bit Operating System", Burges, H.W.Sams, ISBN 0-627-30655-7
>>This is an interesting book because it is not about Windows
>>and it's not about Linux. It's about a roll-your-own Operating
>>System. It even comes with a CD and you can boot a home-grown
>>system on your PC.
>>
>>Now, I don't care if you studied Operating Systems in College.
>>In fact, that just might be the reason why you don't know what
>>you are talking about. Most such college courses in this subject
>>are absolute crap, written by Masters candidates who learned
>>a bunch of words and coined a few of their own.
>>
>>After you learn about Operating Systems, then you might be able
>>to add some new capability to Linux. You know, they accept patches
>>here. If you've got a better way, you make a patch and we'll all
>>try it. It's really that simple.
>>
>>Cheers,
>>Dick Johnson
>>Penguin : Linux version 2.4.22 on an i686 machine (794.73 BogoMips).
>>            Note 96.31% of all statistics are fiction.
>>    
>>
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml/
>  
>

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1199 bytes --]

On Thu, 04 Sep 2003 22:51:38 BST, James Clark said:

> FUD. It mostly works, sometimes it doesn't, but in total the number of 
> working hours of PRODUCTIVE use from it is many orders of magnitude greater. 
> Multiple the number of Windows users in the world by their working time and 
> then do the same for Linux!

So if 500 million people are productive 60% of the time and hosed 40% of the
time, and 5 million people are productive 95% of the time, the 60/40 model is
better because 60% of 500M is more than 95% of 5M?

What's wrong with this picture?

> hence the OS could escape the niche box it currently is in. Please ask Joe 
> User how he feels about rebuilding his whole OS to add IP6 support to an 
> existing stable system etc.

Ask Joe User how he feels about NOT being able to add IPv6 support to
his existing system until his vendor says they'll do it for him, and then
look at when Linux had support.

http://www.ipv6.org/impl/linux.html
http://www.ipv6.org/impl/windows.html

And most important, google around for +ipv6 +"craig metz", and look at when *he*
did the IPv6 work - and you were free to put the patches on your system as
soon as he posted them.

Now as you were saying?

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[James Clark]

Valdis Kletnieks wrote:

> So if 500 million people are productive 60% of the time and hosed 40% of
> the time, and 5 million people are productive 95% of the time, the 60/40
> model is better because 60% of 500M is more than 95% of 5M?

This is a good example of the kind of rubbish that is sometimes talked around 
here. I've lost count of the number of times I've heard the 'Windows is SO 
unstable argument' it almost seems like a religion. I would agree with what 
you have said if Windows was actually unusable 40% of the time. Do you really 
believe this figure? In reality it is much better than that as plainly the 
majority of the WORLD are using it. I love Linux but I also use Windows. 
Sorry to break your delusion, it ain't that bad.

> Ask Joe User how he feels about NOT being able to add IPv6 support to
> his existing system until his vendor says they'll do it for him, and then
> look at when Linux had support.

Its very true that in the Windows world you have to wait for Micro$oft 
sometimes, it is even true that they probably hold back features so that can 
put them in the next release and get you to pay for them. Why is this any 
worse than expecting Joe User, who is a 'user' and not a 'developer' to 
rebuild the most important bits of his OS and risk breaking the whole lot.

This is not about Windows v Linux so please stop compraring what I have 
proposed to Windows. This debate should be about Performance v Usability. 
Source interfaces have ultimate performance, nobody has suggested, yet, that 
they are easier for Joe User than a binary interface.

James



On Friday 05 Sep 2003 6:52 pm, you wrote:
> On Thu, 04 Sep 2003 22:51:38 BST, James Clark said:
> > FUD. It mostly works, sometimes it doesn't, but in total the number of
> > working hours of PRODUCTIVE use from it is many orders of magnitude
> > greater. Multiple the number of Windows users in the world by their
> > working time and then do the same for Linux!
>
> So if 500 million people are productive 60% of the time and hosed 40% of
> the time, and 5 million people are productive 95% of the time, the 60/40
> model is better because 60% of 500M is more than 95% of 5M?
>
> What's wrong with this picture?
>
> > hence the OS could escape the niche box it currently is in. Please ask
> > Joe User how he feels about rebuilding his whole OS to add IP6 support to
> > an existing stable system etc.
>
> Ask Joe User how he feels about NOT being able to add IPv6 support to
> his existing system until his vendor says they'll do it for him, and then
> look at when Linux had support.
>
> http://www.ipv6.org/impl/linux.html
> http://www.ipv6.org/impl/windows.html
>
> And most important, google around for +ipv6 +"craig metz", and look at when
> *he* did the IPv6 work - and you were free to put the patches on your
> system as soon as he posted them.
>
> Now as you were saying?

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Martin Schlemmer]

On Fri, 2003-09-05 at 20:31, James Clark wrote:
> Valdis Kletnieks wrote:
> 
> > So if 500 million people are productive 60% of the time and hosed 40% of
> > the time, and 5 million people are productive 95% of the time, the 60/40
> > model is better because 60% of 500M is more than 95% of 5M?
> 
> This is a good example of the kind of rubbish that is sometimes talked around 
> here. I've lost count of the number of times I've heard the 'Windows is SO 
> unstable argument' it almost seems like a religion. I would agree with what 
> you have said if Windows was actually unusable 40% of the time. Do you really 
> believe this figure? In reality it is much better than that as plainly the 
> majority of the WORLD are using it. I love Linux but I also use Windows. 
> Sorry to break your delusion, it ain't that bad.
> 
> > Ask Joe User how he feels about NOT being able to add IPv6 support to
> > his existing system until his vendor says they'll do it for him, and then
> > look at when Linux had support.
> 
> Its very true that in the Windows world you have to wait for Micro$oft 
> sometimes, it is even true that they probably hold back features so that can 
> put them in the next release and get you to pay for them. Why is this any 
> worse than expecting Joe User, who is a 'user' and not a 'developer' to 
> rebuild the most important bits of his OS and risk breaking the whole lot.
> 
> This is not about Windows v Linux so please stop compraring what I have 
> proposed to Windows. This debate should be about Performance v Usability. 
> Source interfaces have ultimate performance, nobody has suggested, yet, that 
> they are easier for Joe User than a binary interface.
> 

Right.  Not trying to get involved in this mess [8)], but there
was a few other comments that was less linux/windows, but more
valid ... what about responding to them?  Could be interesting
to hear what you have to say ...


> James
> 
> 
> 
> On Friday 05 Sep 2003 6:52 pm, you wrote:
> > On Thu, 04 Sep 2003 22:51:38 BST, James Clark said:
> > > FUD. It mostly works, sometimes it doesn't, but in total the number of
> > > working hours of PRODUCTIVE use from it is many orders of magnitude
> > > greater. Multiple the number of Windows users in the world by their
> > > working time and then do the same for Linux!
> >
> > So if 500 million people are productive 60% of the time and hosed 40% of
> > the time, and 5 million people are productive 95% of the time, the 60/40
> > model is better because 60% of 500M is more than 95% of 5M?
> >
> > What's wrong with this picture?
> >
> > > hence the OS could escape the niche box it currently is in. Please ask
> > > Joe User how he feels about rebuilding his whole OS to add IP6 support to
> > > an existing stable system etc.
> >
> > Ask Joe User how he feels about NOT being able to add IPv6 support to
> > his existing system until his vendor says they'll do it for him, and then
> > look at when Linux had support.
> >
> > http://www.ipv6.org/impl/linux.html
> > http://www.ipv6.org/impl/windows.html
> >
> > And most important, google around for +ipv6 +"craig metz", and look at when
> > *he* did the IPv6 work - and you were free to put the patches on your
> > system as soon as he posted them.
> >
> > Now as you were saying?
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
-- 
Martin Schlemmer

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Dale P. Smith]

James Clark wrote:
> Valdis Kletnieks wrote:
> 
> 
>>So if 500 million people are productive 60% of the time and hosed 40% of
>>the time, and 5 million people are productive 95% of the time, the 60/40
>>model is better because 60% of 500M is more than 95% of 5M?
> 
> 
> This is a good example of the kind of rubbish that is sometimes talked around 
> here. I've lost count of the number of times I've heard the 'Windows is SO 
> unstable argument' it almost seems like a religion. I would agree with what 
> you have said if Windows was actually unusable 40% of the time. Do you really 
> believe this figure? In reality it is much better than that as plainly the 
> majority of the WORLD are using it. I love Linux but I also use Windows. 
> Sorry to break your delusion, it ain't that bad.

The only windows system that I have seen that was reliable (unless the 
people there were lying to me) was a big ibm netfinity system.  I 
believe it was stable because it didn't use any extra harware or drivers 
except what ibm had installed on the box.

What I'm saying is that most of the problems with windows are from 
flakey, half-baked drivers and dll's form third parties.  I've got no 
hard numbers, just my personal experiences.

Binary onlyt drivers are bad.  Source drivers are good.  THat's just the 
way it is.

-Dale

-- 
Dale P. Smith
dsmith at actron dot com

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Stan Bubrouski]

Dale P. Smith wrote:

>
> The only windows system that I have seen that was reliable (unless the 
> people there were lying to me) was a big ibm netfinity system.  I 
> believe it was stable because it didn't use any extra harware or 
> drivers except what ibm had installed on the box.
>
I dunno I had Win2k running with only 2 crashes for 3 years on a
Dell OptiPlex GX300 866MHz SMP system.  Believe it or not for
the first 2 years I never had a full system crash.  I left it on for months
at a time.  That doesn't mean windows is stable though.  It means
for my particular setup it was.  I'm probably one of the few people
I know on campus who haven't had to had to format there windows
and start over several times.   The biggest problem is the windows
registry, once it becomes corrupt unless you have a backup you've
lost all your settings.  This is horrible by design.  Any minor corruption
or registry permissions problems can cause a system to go down
faster than Jennifer Lopez.

> What I'm saying is that most of the problems with windows are from 
> flakey, half-baked drivers and dll's form third parties.  I've got no 
> hard numbers, just my personal experiences.
>
All my problems have been flakey, half-baked drivers and DLLs from
Microsoft, so I cannot relate.

> Binary onlyt drivers are bad.  Source drivers are good.  THat's just 
> the way it is.
>
Agreed.

> -Dale
>

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Richard B. Johnson]

On Fri, 5 Sep 2003, James Clark wrote:

> Valdis Kletnieks wrote:
>
> > So if 500 million people are productive 60% of the time and hosed 40% of
> > the time, and 5 million people are productive 95% of the time, the 60/40
> > model is better because 60% of 500M is more than 95% of 5M?
>
> This is a good example of the kind of rubbish that is sometimes talked around
> here. I've lost count of the number of times I've heard the 'Windows is SO
> unstable argument' it almost seems like a religion. I would agree with what
> you have said if Windows was actually unusable 40% of the time. Do you really
> believe this figure? In reality it is much better than that as plainly the
> majority of the WORLD are using it. I love Linux but I also use Windows.
> Sorry to break your delusion, it ain't that bad.
>

I sure wish it was rubbish. In my other life I write
music. So, I bought Cakewalk Software so I could record
and edit music I play on my MIDI Grand Piano. Unfortunately
this only worked with Windows.

I wanted to have the most reliable machine possible so I
installed windows in a machine that I had been running Linux
on for about two years. It was (is) a dual Pentium 400 MHz machine
with a 100 MHz front-side bus. This is a Tyan "Thunder" board.
This is not the latest-and-greatest. Just a good "old" reliable
machine. The machine had two SCSI disks with an Adaptec controller.

Windows installed fine. It even seemed to work. I recorded and
edited many compositions. I also backed up my files to a SCSI
tape using the Windows backup utility.

After several months of use, the machine would not boot anymore.
I had to reinstall windows from scratch since the partition was
corrupt and the distribution CD would not fix or install anything
over it. Since I didn't want to lose everything on the disk, I
bought another one, thinking I could always copy my stuff from
the original. Guess again. The original disk was so destroyed
that Linux wouldn't even recognize it. Even as a raw device I was
unable to recover any of my composition data.

Not to worry. I still have the backups. Guess again, Windows
would not read the backup data. I had just lost several months
of my life and, incidentally, some piano playing by another
pianist who will become famous someday. All lost. All gone.

I eventually replaced all my compositions, except those played
by the other pianist. This means that I had to practice over
40 compositions over and over again, then record them again.
This is a lot of "hurt" over many months.

Then the same thing happened again! The machine would no longer
boot and the file-systems were totally trashed. Also, I had bought
an external Fire-wire drive to which I wrote the backup files.
Again, after completely installing everything again, I was unable
to recover any of my data from the backup files.

To make an long-story short, the "solution", proposed by Microsoft
was to remove one of the CPUs. They said; "Windows is too powerful.
With that extra CPU, you are over-powering the machine."

This is not an isolated incident. Every Engineer I have ever worked
with (or even talked to) has similar horror stories. Every Writer
that I know of who has attempted to use Windows to keep their
life-long dreams alive, has similar horror stories. Every Artist
that I know, who used Windows has similar stories. They have all
migrated to Apple. They were not of the "Apple Generation" either.
They all started out using Windows because that's what "everybody"
used.  They went to Apple because it doesn't destroy their work.

Most persons who use Linux have similar horror stories about
Windows. The difference, here, being that they thought that
no widely-used Operating System could be as bad as they found
it to be. Instead, they for the most part, though that their
problems were isolated incidents. Once they started communicating,
it became obvious that practically everybody has problems with
Windows. In the days where VAXen were the "Engineering" computers,
we had 200 Engineers, two computers, and one System Manager.
Now, those 200 Engineers require 30 "Windows Administrators"
for support, just to be able to share project directories.

They spend their time reinstalling Windows Software.

Cakewalk recently released an Apple version of "Home Studio". I'm
going to buy and Apple and convert the Windows machine back to
Linux. I wish they had ported their stuff to Linux. I would still
have some hair left.

So, if you think Windows "works", you just haven't any experience.
In a year or two, maximum, if you are trying to use a Windows
machine for any serious work, you will probably be planning a trip
to Redmond, carrying an axe.

When you talk about 40% or even 10% failure rate, you are talking
nonsense. A computing machine must never have any failures that
an end-user could detect. It's not quite six-sigma, but to the
end-user it should seem like that. Any real operating system
will not fail as long as the hardware doesn't fail. No version
of Windows qualifies.

Cheers,
Dick Johnson
Penguin : Linux version 2.4.22 on an i686 machine (794.73 BogoMips).
            Note 96.31% of all statistics are fiction.

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[James Clark]

My original thread was called 'Linux Kernel Performance v Usability'. This has 
nothing to do with Windows. 

James


On Friday 05 Sep 2003 8:59 pm, you wrote:
> On Fri, 5 Sep 2003, James Clark wrote:
> > Valdis Kletnieks wrote:
> > > So if 500 million people are productive 60% of the time and hosed 40%
> > > of the time, and 5 million people are productive 95% of the time, the
> > > 60/40 model is better because 60% of 500M is more than 95% of 5M?
> >
> > This is a good example of the kind of rubbish that is sometimes talked
> > around here. I've lost count of the number of times I've heard the
> > 'Windows is SO unstable argument' it almost seems like a religion. I
> > would agree with what you have said if Windows was actually unusable 40%
> > of the time. Do you really believe this figure? In reality it is much
> > better than that as plainly the majority of the WORLD are using it. I
> > love Linux but I also use Windows. Sorry to break your delusion, it ain't
> > that bad.
>
> I sure wish it was rubbish. In my other life I write
> music. So, I bought Cakewalk Software so I could record
> and edit music I play on my MIDI Grand Piano. Unfortunately
> this only worked with Windows.
>
> I wanted to have the most reliable machine possible so I
> installed windows in a machine that I had been running Linux
> on for about two years. It was (is) a dual Pentium 400 MHz machine
> with a 100 MHz front-side bus. This is a Tyan "Thunder" board.
> This is not the latest-and-greatest. Just a good "old" reliable
> machine. The machine had two SCSI disks with an Adaptec controller.
>
> Windows installed fine. It even seemed to work. I recorded and
> edited many compositions. I also backed up my files to a SCSI
> tape using the Windows backup utility.
>
> After several months of use, the machine would not boot anymore.
> I had to reinstall windows from scratch since the partition was
> corrupt and the distribution CD would not fix or install anything
> over it. Since I didn't want to lose everything on the disk, I
> bought another one, thinking I could always copy my stuff from
> the original. Guess again. The original disk was so destroyed
> that Linux wouldn't even recognize it. Even as a raw device I was
> unable to recover any of my composition data.
>
> Not to worry. I still have the backups. Guess again, Windows
> would not read the backup data. I had just lost several months
> of my life and, incidentally, some piano playing by another
> pianist who will become famous someday. All lost. All gone.
>
> I eventually replaced all my compositions, except those played
> by the other pianist. This means that I had to practice over
> 40 compositions over and over again, then record them again.
> This is a lot of "hurt" over many months.
>
> Then the same thing happened again! The machine would no longer
> boot and the file-systems were totally trashed. Also, I had bought
> an external Fire-wire drive to which I wrote the backup files.
> Again, after completely installing everything again, I was unable
> to recover any of my data from the backup files.
>
> To make an long-story short, the "solution", proposed by Microsoft
> was to remove one of the CPUs. They said; "Windows is too powerful.
> With that extra CPU, you are over-powering the machine."
>
> This is not an isolated incident. Every Engineer I have ever worked
> with (or even talked to) has similar horror stories. Every Writer
> that I know of who has attempted to use Windows to keep their
> life-long dreams alive, has similar horror stories. Every Artist
> that I know, who used Windows has similar stories. They have all
> migrated to Apple. They were not of the "Apple Generation" either.
> They all started out using Windows because that's what "everybody"
> used.  They went to Apple because it doesn't destroy their work.
>
> Most persons who use Linux have similar horror stories about
> Windows. The difference, here, being that they thought that
> no widely-used Operating System could be as bad as they found
> it to be. Instead, they for the most part, though that their
> problems were isolated incidents. Once they started communicating,
> it became obvious that practically everybody has problems with
> Windows. In the days where VAXen were the "Engineering" computers,
> we had 200 Engineers, two computers, and one System Manager.
> Now, those 200 Engineers require 30 "Windows Administrators"
> for support, just to be able to share project directories.
>
> They spend their time reinstalling Windows Software.
>
> Cakewalk recently released an Apple version of "Home Studio". I'm
> going to buy and Apple and convert the Windows machine back to
> Linux. I wish they had ported their stuff to Linux. I would still
> have some hair left.
>
> So, if you think Windows "works", you just haven't any experience.
> In a year or two, maximum, if you are trying to use a Windows
> machine for any serious work, you will probably be planning a trip
> to Redmond, carrying an axe.
>
> When you talk about 40% or even 10% failure rate, you are talking
> nonsense. A computing machine must never have any failures that
> an end-user could detect. It's not quite six-sigma, but to the
> end-user it should seem like that. Any real operating system
> will not fail as long as the hardware doesn't fail. No version
> of Windows qualifies.
>
> Cheers,
> Dick Johnson
> Penguin : Linux version 2.4.22 on an i686 machine (794.73 BogoMips).
>             Note 96.31% of all statistics are fiction.

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Mike Fedyk]

On Fri, Sep 05, 2003 at 07:31:09PM +0100, James Clark wrote:
> This is not about Windows v Linux so please stop compraring what I have 
> proposed to Windows. This debate should be about Performance v Usability. 
> Source interfaces have ultimate performance, nobody has suggested, yet, that 
> they are easier for Joe User than a binary interface.

Here is one thing we don't have standardized across the entire Linux
distribution landscape.

What you need is a project that will take the top 10 distributions, and do
this however each distribution does their thing:

 o identify the current kernel running (you're going to use the kernel
   you're running, right?)
  
 o download the kernel source for the running kernel

 o install the source in some temporary location

 o compile against the downloaded kernel source

 o install the module under /lib/modules

 o load the module (with the corect optional parameters)

Now what we need is someone to create the project, and get the community to
point to that project for source available external driver modules.

And look!  No kernel modifications!

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1520 bytes --]

On Fri, 05 Sep 2003 19:31:09 BST, James Clark said:

> > So if 500 million people are productive 60% of the time and hosed 40% of
> > the time, and 5 million people are productive 95% of the time, the 60/40
> > model is better because 60% of 500M is more than 95% of 5M?
> 
> This is a good example of the kind of rubbish that is sometimes talked around
 
> here. I've lost count of the number of times I've heard the 'Windows is SO 
> unstable argument' it almost seems like a religion. I would agree with what 
> you have said if Windows was actually unusable 40% of the time. 

What you said:

 > FUD. It mostly works, sometimes it doesn't, but in total the number of
> > working hours of PRODUCTIVE use from it is many orders of magnitude
> > greater. Multiple the number of Windows users in the world by their
> > working time and then do the same for Linux!

Oh.. so now you're complaining that I pointed out that even a system that's
down 40% of the time meets *YOUR* criteria if enough more people run it?

In fact, if the number of Windows users is 100 times the number of Linux users,
and Linux is 98% reliable, then Windows only needs to make about 2% reliable
to win by your criteria.  OK, you want to complain about the 100?  Make it
500M windows and 50M linux, and Linux 98% reliable, then Windows only
needs to make 10% uptime to win by your criteria.

My point is that your criteria of "total aggregate uptime" doesn't prove anything
between diddly and squat about the actual reliability of the system.


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Bernd Eckenfels]

In article <200309052115.h85LFtXh004015@turing-police.cc.vt.edu> you wrote:
> My point is that your criteria of "total aggregate uptime" doesn't prove anything
> between diddly and squat about the actual reliability of the system.

Even if this is a not very on-topic discussion, and especially because I do
not agree about Windows beeing unstable at all:

- how about agregating the downtimes as the cummulative lost time by
instability.

Greetings
Bernd
-- 
eckes privat - http://www.eckes.org/
Project Freefire - http://www.freefire.org/

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Timothy Miller]

I just have one quick question about all of this:

People mention that driver interfaces don't change much in stable 
releases, but if memory serves, symbol versioning information changes 
with each minor release, requiring a recompile of modules.

Would it be possible to have a driver module which can be dropped into, 
say, 2.6.17 that can also be dropped into 2.6.18 as long as the 
interface doesn't change?

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Richard B. Johnson]

On Wed, 10 Sep 2003, Timothy Miller wrote:

> I just have one quick question about all of this:
>
> People mention that driver interfaces don't change much in stable
> releases, but if memory serves, symbol versioning information changes
> with each minor release, requiring a recompile of modules.
>
> Would it be possible to have a driver module which can be dropped into,
> say, 2.6.17 that can also be dropped into 2.6.18 as long as the
> interface doesn't change?
>

Short answer, YES. Anything that can be done is possible. The
problem is that different kernel versions end up with different
structure members, etc. So, you can't use code for 2.2.xxx in
2.4.xx because, amongst other things, the first element in
'struct file_operations' was added and the others moved up.

Now, you can make a different module interface that maintains
a compatibility level ABI. This has been discussed. Unfortunately,
this adds code in the execution path. This extra code gets
executed every time the module code is accessed. The result being
that the module can't possibly operate as fast as it would if
there were no such compatibility layer(s). It might be "good enough",
but it is unlikely that the module contributors/maintainers would
allow such an interface because the loss of performance is measurable
and there has been no requirement to trade-off performance for
anything (your and my convenience doesn't count, those are not
technical issues).


Cheers,
Dick Johnson
Penguin : Linux version 2.4.22 on an i686 machine (794.73 BogoMips).
            Note 96.31% of all statistics are fiction.

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[James Clark]

Has anyone ever done any work to quantify what the loss in performance might 
be with a binary interface? 

James

On Wednesday 10 Sep 2003 9:48 pm, Richard B. Johnson wrote:
> On Wed, 10 Sep 2003, Timothy Miller wrote:
> > I just have one quick question about all of this:
> >
> > People mention that driver interfaces don't change much in stable
> > releases, but if memory serves, symbol versioning information changes
> > with each minor release, requiring a recompile of modules.
> >
> > Would it be possible to have a driver module which can be dropped into,
> > say, 2.6.17 that can also be dropped into 2.6.18 as long as the
> > interface doesn't change?
>
> Short answer, YES. Anything that can be done is possible. The
> problem is that different kernel versions end up with different
> structure members, etc. So, you can't use code for 2.2.xxx in
> 2.4.xx because, amongst other things, the first element in
> 'struct file_operations' was added and the others moved up.
>
> Now, you can make a different module interface that maintains
> a compatibility level ABI. This has been discussed. Unfortunately,
> this adds code in the execution path. This extra code gets
> executed every time the module code is accessed. The result being
> that the module can't possibly operate as fast as it would if
> there were no such compatibility layer(s). It might be "good enough",
> but it is unlikely that the module contributors/maintainers would
> allow such an interface because the loss of performance is measurable
> and there has been no requirement to trade-off performance for
> anything (your and my convenience doesn't count, those are not
> technical issues).
>
>
> Cheers,
> Dick Johnson
> Penguin : Linux version 2.4.22 on an i686 machine (794.73 BogoMips).
>             Note 96.31% of all statistics are fiction.
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Greg KH]

On Thu, Sep 11, 2003 at 12:22:16AM +0100, James Clark wrote:
> Has anyone ever done any work to quantify what the loss in performance might 
> be with a binary interface? 

See the UDI results.

greg k-h

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Timothy Miller]

Richard B. Johnson wrote:
> On Wed, 10 Sep 2003, Timothy Miller wrote:
> 
> 
>>I just have one quick question about all of this:
>>
>>People mention that driver interfaces don't change much in stable
>>releases, but if memory serves, symbol versioning information changes
>>with each minor release, requiring a recompile of modules.
>>
>>Would it be possible to have a driver module which can be dropped into,
>>say, 2.6.17 that can also be dropped into 2.6.18 as long as the
>>interface doesn't change?
>>
> 
> 
> Short answer, YES. Anything that can be done is possible. The
> problem is that different kernel versions end up with different
> structure members, etc. So, you can't use code for 2.2.xxx in
> 2.4.xx because, amongst other things, the first element in
> 'struct file_operations' was added and the others moved up.

That's all fine and dandy.  When the kernel changes its interface, then 
you have to recompile (or rewrite) drivers.  No problem.  I'm just 
trying to avoid having to recompile drivers if the interface DOESN'T change.

> 
> Now, you can make a different module interface that maintains
> a compatibility level ABI. This has been discussed. Unfortunately,
> this adds code in the execution path. This extra code gets
> executed every time the module code is accessed. The result being
> that the module can't possibly operate as fast as it would if
> there were no such compatibility layer(s). It might be "good enough",
> but it is unlikely that the module contributors/maintainers would
> allow such an interface because the loss of performance is measurable
> and there has been no requirement to trade-off performance for
> anything (your and my convenience doesn't count, those are not
> technical issues).

I am not interested in adding additional layers of abstraction.  At 
least not here.  I do it plenty of other places, but that's not 
important right now.  If someone else wants to make an abstraction layer 
(which seems to have been done here and there), then that's just fine, 
and I may or may not use it.

My point is that I'm not advocating any of the kruft associated with 
backward-compatible interfaces.  I'm advocating not having to recompile 
only in the cases where the interface DOES NOT change.

Why?  Because there are some advantages to being able to say that this 
one module can be dropped into any box running, for instance, 2.6.12 
through 2.6.16, while the next module is used for 2.6.17 thru 2.6.22, etc.

For distributions, this can be helpful for drivers which are not in the 
mainline kernel.  So, I'm not trying to necessarily find a way to help 
binary-only drivers (although it would, to some extent).

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Tim Hockin]

On Fri, Sep 12, 2003 at 04:51:35PM -0400, Timothy Miller wrote:
> Why?  Because there are some advantages to being able to say that this 
> one module can be dropped into any box running, for instance, 2.6.12 
> through 2.6.16, while the next module is used for 2.6.17 thru 2.6.22, etc.

There are a SLEW of options that make this hard.

CONFIG_SMP - spinlocks get compiled in or not

CONFIG_SPINLOCK_DEBUGGING - changes the size of a spinlock and the behavior
of spin_lock() and friend

All the HIGHMEM and various kernel/user split option - change the kernel
virtual addresses and offsets

And these are just a few.  The thing is, there isn't any way to catalog them
all, because at any point, someone can add a CONFIG_FOO_DEBUG which changes
the size of a struct foo and foo_op().  Then any module which uses a foo has
to ALSO have FOO_DEBUG on.  Modules and kernels must match up on ALL the
options (or at least, all the ones that cross the kernel-module boundary).

This actually bit me just last week.  I turn on spinlock debugging, and a
module Oopses.  I didn't recompile modules.

It sucks.  I agree.  Maybe there is an elegant way to checksum each
functional interface and each datatype and typedef and each argument to each
funtion and make it such that IFF they are all the same, the module will
load.  It's just not pretty.

-- 
Notice that as computers are becoming easier and easier to use,
suddenly there's a big market for "Dummies" books.  Cause and effect,
or merely an ironic juxtaposition of unrelated facts?

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Richard B. Johnson]

On Fri, 12 Sep 2003, Timothy Miller wrote:

>
>
> Richard B. Johnson wrote:
> > On Wed, 10 Sep 2003, Timothy Miller wrote:
> >
> >
> >>I just have one quick question about all of this:
> >>
> >>People mention that driver interfaces don't change much in stable
> >>releases, but if memory serves, symbol versioning information changes
> >>with each minor release, requiring a recompile of modules.
> >>
> >>Would it be possible to have a driver module which can be dropped into,
> >>say, 2.6.17 that can also be dropped into 2.6.18 as long as the
> >>interface doesn't change?
> >>
> >
> >
> > Short answer, YES. Anything that can be done is possible. The
> > problem is that different kernel versions end up with different
> > structure members, etc. So, you can't use code for 2.2.xxx in
> > 2.4.xx because, amongst other things, the first element in
> > 'struct file_operations' was added and the others moved up.
>
> That's all fine and dandy.  When the kernel changes its interface, then
> you have to recompile (or rewrite) drivers.  No problem.  I'm just
> trying to avoid having to recompile drivers if the interface DOESN'T change.
>
> >
> > Now, you can make a different module interface that maintains
> > a compatibility level ABI. This has been discussed. Unfortunately,
> > this adds code in the execution path. This extra code gets
> > executed every time the module code is accessed. The result being
> > that the module can't possibly operate as fast as it would if
> > there were no such compatibility layer(s). It might be "good enough",
> > but it is unlikely that the module contributors/maintainers would
> > allow such an interface because the loss of performance is measurable
> > and there has been no requirement to trade-off performance for
> > anything (your and my convenience doesn't count, those are not
> > technical issues).
>
> I am not interested in adding additional layers of abstraction.  At
> least not here.  I do it plenty of other places, but that's not
> important right now.  If someone else wants to make an abstraction layer
> (which seems to have been done here and there), then that's just fine,
> and I may or may not use it.
>
> My point is that I'm not advocating any of the kruft associated with
> backward-compatible interfaces.  I'm advocating not having to recompile
> only in the cases where the interface DOES NOT change.
>

In those cases, you simply use "-f" on the `insmod` command line
to force the loading of a module. We have that capability now.
But, the offsets of things your module(s) access can, read will,
be incompatible if the version changed. You "takes you chances and
cross you fingers...." if you use the "-f" option.

Cheers,
Dick Johnson
Penguin : Linux version 2.4.22 on an i686 machine (794.73 BogoMips).
            Note 96.31% of all statistics are fiction.

RE: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Chad Kitching]

From: Mike Fedyk [mailto:mfedyk@matchmail.com]
> Here is one thing we don't have standardized across the entire Linux
> distribution landscape.
> 
> What you need is a project that will take the top 10 
> distributions, and do
> this however each distribution does their thing:
> 
>  o identify the current kernel running (you're going to use the kernel
>    you're running, right?)

Not to mention on boot-up check to make sure the module still loads 
without warnings on the current kernel (or make sure the module exists 
in the current /lib/modules directory.
   
>  o download the kernel source for the running kernel

Problem: Most distributors modify their kernel somewhat.  Some enough 
to cause binary module incompatibility with the 'stock' kernel.  
Matching running kernel and source code kernel would be tricky, to
say the least.
 
>  o install the source in some temporary location

Why not just make the includes directory get installed somewhere.  
Somewhere like /lib/modules/`uname -r`/build/includes (especially since
make install puts a symlink at /lib/modules/`uname -r`/build anyway)
You also need to prep the extracted kernel with the proper .config, etc.
which isn't always in the source package from some distributors.

>  o compile against the downloaded kernel source
> 
>  o install the module under /lib/modules
> 
>  o load the module (with the corect optional parameters)

The biggest problem is people not having installed the C compiler, and 
related tools.  Or having not installed the kernel headers matching 
their version of the kernel.

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Mike Fedyk]

On Fri, Sep 05, 2003 at 03:53:27PM -0500, Chad Kitching wrote:
> 
> From: Mike Fedyk [mailto:mfedyk@matchmail.com]
> > Here is one thing we don't have standardized across the entire Linux
> > distribution landscape.
> > 
> > What you need is a project that will take the top 10 
> > distributions, and do
> > this however each distribution does their thing:
> > 
> >  o identify the current kernel running (you're going to use the kernel
> >    you're running, right?)
> 
> Not to mention on boot-up check to make sure the module still loads 
> without warnings on the current kernel (or make sure the module exists 
> in the current /lib/modules directory.
>    

Not a problem. and /lib/modules is /lib/modules/`uname -r`
You have to compile the driver for the new kernel.  see below

> >  o download the kernel source for the running kernel
> 
> Problem: Most distributors modify their kernel somewhat.  Some enough 
> to cause binary module incompatibility with the 'stock' kernel.  
> Matching running kernel and source code kernel would be tricky, to
> say the least.
>  

That is why it has to be modified to detect and work with the 10 most
popular distributions.  You work with the distribution kernel...

> >  o install the source in some temporary location
> 
> Why not just make the includes directory get installed somewhere.  
> Somewhere like /lib/modules/`uname -r`/build/includes (especially since
> make install puts a symlink at /lib/modules/`uname -r`/build anyway)
> You also need to prep the extracted kernel with the proper .config, etc.
> which isn't always in the source package from some distributors.
> 

Because that takes up more space, and some people don't like that.  Though
the distributions could make it an option to do that on install.  It's up to
the individual distribution though...

> >  o compile against the downloaded kernel source
> > 
> >  o install the module under /lib/modules

should have been /lib/modules/`uname -r`

> > 
> >  o load the module (with the corect optional parameters)
> 
> The biggest problem is people not having installed the C compiler, and 
> related tools.  Or having not installed the kernel headers matching 
> their version of the kernel.
> 

 o identify gcc generation (2.95x, 3.0x, 3.1x, etc) to build current kernel
 
 o install gcc version to match

There are tools already to add patches to kernels in debian (wheather it's a
debian kernel or not).  We just need to have something the other way around.
No need to patch the kernel, just compile against it.

I say that it should adapt to the individual distribution because it takes
so much longer to make a standard, and that can always come later, and the
project can adjust to the new standard (instead of complaining to each
distribution whenever they change their kernel packaging technique).

RE: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Chad Kitching]

From: James Clark [mailto:jimwclark@ntlworld.com]
> 
> Thank you for this (and the few other) sensible appraisal of 
> my 'proposal'. 
> 
> I'm very surprised by the number of posts that have ranted 
> about Open/Close source, GPL/taint issues etc. This is not
> about source code it is about making Linux usable by the 
> masses. It may be technically superior to follow the current 
> model, but if the barrier to entry is very high (and it is!) 
> then the project will continue to be a niche project. A 
> binary model doesn't alter the community or the benefits 
> of public source code. I agree that it is an extra interface 
> and will carry a performance hit - I think this is worth it. 
> Windows has many faults but drivers are often compatible across 
> major releases and VERY compatible across minor releases. It 
> is no accident that it has 90% of the desktop market. If we are 
> going to improve this situation this issue MUST be confronted.

I'll admit, I don't see how a binary only interface helps easy of use.  For the most part, common users in the Windows world can't handle installing a driver any better than they would be able to in Linux.

Now, before going any further with your proposal, look at http://www.projectudi.org/, who were commited to doing exactly what you propose with even more ambition dreams of cross OS compatibility.  It was a Caldera/SCO project, so it's more than likely dead on the Linux side, but you can see the technical description, specifications, and even source code for this type of  endevour.

You can read the some of the UDI debate at:
http://lkml.org/lkml/1998/9/18/37
http://lkml.org/lkml/1998/8/31/63

The problems with such an API are very numerous.  For an example, just look at Windows NT 4.0 versus Windows 2000 versus Windows XP or Windows 95->Windows 98->Windows 98SE->Windows ME.  In both those cases, the individual revisions have the same basic driver model, but added features in each made drivers from the previous one more or less incompatible with the newly released OS.  Now, lets assume a project like UDI was accepted into Linux.  What would happen when technology evolves, and the old stuff becomes obsolete?  Will Linux have to always refer to devices on fancy new buses as if they were on old PCI buses?  What happens when some core OS bug fix breaks three or four different drivers?  Will Linux have to incorporate bug workarounds for those defective drivers?  What will happen when the binary API for device drivers starts making the Windows DDK look simple?  The Linux module API is very, very simple compared to most other OS's driver API.  Someone can learn to make their first kernel driver in a few hours, and even have it talking to hardware within that time.

Source code for drivers is an advantage for Linux.  It ensures that anyone can have a chance to fix quirky bugs that are getting in the way of them using their computer to it's fullest.  I used to have problems with some particular 3c905 cards with Linux 2.4.  If I had been using binary drivers, my only recourse would be to try and pester the vendor (3COM) for a fix.  Instead, I found a trivial patch to the kernel that I applied, and recompiled, and got the cards working without a hitch.  You could try to argue that the commerical released driver would not have a problem like this, but it would likely also come with a compatibility list that only said it works on Red Hat 6.2, and Red Hat 7.0, and with me running Debian, I'd be out of luck -- no support, even if it SHOULD work.

The reality is that most people will simply use whatever their distributor has given them.  If it's Red Hat, they'll use whatever came with their Red Hat CDs.  If it came with a handy CD with install instructions, they might try to install that.  Otherwise, they'll hire a qualified person to do the work for them, exactly as if it were a Windows machine.  There have been no end of the people who insist that Linux is too hard to use because compiling the kernel is too difficult, and the obvious suggestion is then, to not compile the kernel and just use whatever your distributor gave you instead!  Most people won't need to worry about the source code, and may never need to touch it, but the fact that it's there for them, gives them some power that they otherwise would not have.

Driver Model 2 Proposal - Linux Kernel Performance v Usability

[James Clark]

Following my initial post yesterday please find attached my proposal for a 
binary 'plugin' interface:

This is not an attempt to have a Microkernel, or any move away from GNU/OSS 
software. I believe that sometimes the ultimate goals of stability and 
portability get lost in the debate on OSS and desire to allow anyone to 
contribute. It is worth remembering that for every Kernel hacker there must 
be 1000's of plain users. I believe this proposal would lead to better 
software and more people using it.

Proposal
-----------
1. Implement binary kernel 'plugin' interface
2. Over time remove most existing kernel 'drivers' to use new interface - This 
is NOT a Microkernel.
3. Design 'plugin' interface to be extensible as possible and then rarely 
remove support from interface. Extending interface is fine but should be done 
in a considered way to avoid interface bloat. Suggest interface supports 
dependant 'plugins'
4. Allow 'plugins' to be bypassed at boot - perhaps a minimal 'known good' 
list
5. Ultimately, even FS 'plugins' could be created although IPL would be 
required to load these. 
6. Code for Kernel, Interface and 'plugins' would still be GPL. This would not 
prevent the 'tainted' system idea. 

Expected Outcomes
------------------------

1. Make Linux easier to use
2. The ability to replace even very core Kernel components without a restart.
3. Allow faulty 'plugins' to be fixed/replaced in isolation. No other system 
impact.
4. 'Plugins' could create their own interfaces as load time. This would remove 
the need to pre-populate /dev. 
5. Remove need for joe soap user to often recompile Kernel.
6. Remove link between specific module versions and Kernel versions.
7. Reduce need for major Kernel releases. Allow effort to concentrate on 
improving Kernel not maintaining ever increasing Kernel source that includes 
support for the 'Kitchen Sink'
8. Make core Kernel more stable. Less releases and less changes mean less 
bugs. It would be easy to identify offending 'plugin' by simply starting up 
the Kernel with it disabled.
9. Remove need for modules to be maintained in sync with each Kernel thus 
freeing 'module' developers to add improved features or work on new projects.

James

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Andre Hedrick]

James,

You missed a key point, the kernel people do not give a damn about binary
modules and think they are evil.  They do not care if a binary module
works at all.  If they can break it they will.  You are wasting time and
electrons, or just pee'ng in the wind.

The only solution is to created a GPL pre-loading module with all the
GPL_ONLY needed extentions re-exported or externed as to bypass the horse
sh*t.

What I am shocked to see is how people are being nice to you and not
blasting you with a carpet bomb attack.  I guess they are waiting to line
up on me.

Cheers,

Andre



On Wed, 3 Sep 2003, James Clark wrote:

> Following my initial post yesterday please find attached my proposal for a 
> binary 'plugin' interface:
> 
> This is not an attempt to have a Microkernel, or any move away from GNU/OSS 
> software. I believe that sometimes the ultimate goals of stability and 
> portability get lost in the debate on OSS and desire to allow anyone to 
> contribute. It is worth remembering that for every Kernel hacker there must 
> be 1000's of plain users. I believe this proposal would lead to better 
> software and more people using it.
> 
> Proposal
> -----------
> 1. Implement binary kernel 'plugin' interface
> 2. Over time remove most existing kernel 'drivers' to use new interface - This 
> is NOT a Microkernel.
> 3. Design 'plugin' interface to be extensible as possible and then rarely 
> remove support from interface. Extending interface is fine but should be done 
> in a considered way to avoid interface bloat. Suggest interface supports 
> dependant 'plugins'
> 4. Allow 'plugins' to be bypassed at boot - perhaps a minimal 'known good' 
> list
> 5. Ultimately, even FS 'plugins' could be created although IPL would be 
> required to load these. 
> 6. Code for Kernel, Interface and 'plugins' would still be GPL. This would not 
> prevent the 'tainted' system idea. 
> 
> Expected Outcomes
> ------------------------
> 
> 1. Make Linux easier to use
> 2. The ability to replace even very core Kernel components without a restart.
> 3. Allow faulty 'plugins' to be fixed/replaced in isolation. No other system 
> impact.
> 4. 'Plugins' could create their own interfaces as load time. This would remove 
> the need to pre-populate /dev. 
> 5. Remove need for joe soap user to often recompile Kernel.
> 6. Remove link between specific module versions and Kernel versions.
> 7. Reduce need for major Kernel releases. Allow effort to concentrate on 
> improving Kernel not maintaining ever increasing Kernel source that includes 
> support for the 'Kitchen Sink'
> 8. Make core Kernel more stable. Less releases and less changes mean less 
> bugs. It would be easy to identify offending 'plugin' by simply starting up 
> the Kernel with it disabled.
> 9. Remove need for modules to be maintained in sync with each Kernel thus 
> freeing 'module' developers to add improved features or work on new projects.
> 
> James
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Guillaume Morin]

Dans un message du 03 Sep à 10:49, Andre Hedrick écrivait :
> The only solution is to created a GPL pre-loading module with all the
> GPL_ONLY needed extentions re-exported or externed as to bypass the
> horse sh*t.

If you do that, you 

-- 
Guillaume Morin <guillaume@morinfr.org>

      If you cause problems that only you can fix, you'll never be fired.
                            http://www.copyleft.net/

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Andre Hedrick]

On Wed, 3 Sep 2003, Guillaume Morin wrote:

> Dans un message du 03 Sep à 10:49, Andre Hedrick écrivait :
> > The only solution is to created a GPL pre-loading module with all the
> > GPL_ONLY needed extentions re-exported or externed as to bypass the
> > horse sh*t.
> 
> If you do that, you 

I what ?

Create and original work that happens to be open sourced and free and
given away for any binary module vendors to use and operate in linux.
Allow people the choice/option to go open source or closed source in their
product selections.

All I see is a WIN -- WIN for the people using Linux.

Then again, in the beginning it was about having a choice.

Cheers,

Andre

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Guillaume Morin]

Dans un message du 03 Sep à 10:49, Andre Hedrick écrivait :
> The only solution is to created a GPL pre-loading module with all the
> GPL_ONLY needed extentions re-exported or externed as to bypass the
> horse sh*t.

Which would be a violation of the GPL :

What is the difference between "mere aggregation" and "combining two
modules into one program"?  

   Mere aggregation of two programs means putting them side by side on
the same CD-ROM or hard disk. We use this term in the case where they
are separate programs, not parts of a single program. In this case, if
one of the programs is covered by the GPL, it has no effect on the other
program.

    Combining two modules means connecting them together so that they form a
single larger program. If either part is covered by the GPL, the whole
combination must also be released under the GPL--if you can't, or won't, do
that, you may not combine them.

    What constitutes combining two parts into one program? This is a legal
question, which ultimately judges will decide. We believe that a proper
criterion depends both on the mechanism of communication (exec, pipes, rpc,
function calls within a shared address space, etc.) and the semantics of the
communication (what kinds of information are interchanged).

    If the modules are included in the same executable file, they are
definitely combined in one program. If modules are designed to run linked
together in a shared address space, that almost surely means combining them
into one program.

    By contrast, pipes, sockets and command-line arguments are communication
mechanisms normally used between two separate programs. So when they are used
for communication, the modules normally are separate programs. But if the
semantics of the communication are intimate enough, exchanging complex internal
data structures, that too could be a basis to consider the two parts as
combined into a larger program.

-- 
Guillaume Morin <guillaume@morinfr.org>

     Tu veux que les gens réagissent ? Alors commence par réagir (Lofofora)

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Andre Hedrick]

On Wed, 3 Sep 2003, Guillaume Morin wrote:

> Dans un message du 03 Sep à 10:49, Andre Hedrick écrivait :
> > The only solution is to created a GPL pre-loading module with all the
> > GPL_ONLY needed extentions re-exported or externed as to bypass the
> > horse sh*t.
> 
> Which would be a violation of the GPL :
> 
> What is the difference between "mere aggregation" and "combining two
> modules into one program"?  
> 
>    Mere aggregation of two programs means putting them side by side on
> the same CD-ROM or hard disk. We use this term in the case where they
> are separate programs, not parts of a single program. In this case, if
> one of the programs is covered by the GPL, it has no effect on the other
> program.
> 
>     Combining two modules means connecting them together so that they form a
> single larger program. If either part is covered by the GPL, the whole
> combination must also be released under the GPL--if you can't, or won't, do
> that, you may not combine them.
> 
>     What constitutes combining two parts into one program? This is a legal
> question, which ultimately judges will decide. We believe that a proper
> criterion depends both on the mechanism of communication (exec, pipes, rpc,
> function calls within a shared address space, etc.) and the semantics of the
> communication (what kinds of information are interchanged).

You assume wrong.

The Pre-Loader or "freed-symbols" is to return symbols that were once
EXPORT_SYMBOL which are now openly renamed EXPORT_SYMBOL_GPL.

This is open thieft of the kernel API.

So I welcome you and your lawyers to sue me for making a GPL module which
allows free use of the stolen symbols.  Please come to California and lets
party.  Since it is an original work as a new module, regardless if it is
derived or not, being GPL and source available you lose ... Have a nice
day.

Cheers,

Andre

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Greg KH]

On Wed, Sep 03, 2003 at 06:53:01PM +0100, James Clark wrote:
> Following my initial post yesterday please find attached my proposal for a 
> binary 'plugin' interface:
> 
> This is not an attempt to have a Microkernel, or any move away from GNU/OSS 
> software. I believe that sometimes the ultimate goals of stability and 
> portability get lost in the debate on OSS and desire to allow anyone to 
> contribute. It is worth remembering that for every Kernel hacker there must 
> be 1000's of plain users. I believe this proposal would lead to better 
> software and more people using it.
> 
> Proposal
> -----------
> 1. Implement binary kernel 'plugin' interface

And this interface will look like what?
What is wrong with the current kernel API interface?

> 2. Over time remove most existing kernel 'drivers' to use new interface - This 
> is NOT a Microkernel.

"remove"???

> 3. Design 'plugin' interface to be extensible as possible and then rarely 
> remove support from interface. Extending interface is fine but should be done 
> in a considered way to avoid interface bloat. Suggest interface supports 
> dependant 'plugins'
> 4. Allow 'plugins' to be bypassed at boot - perhaps a minimal 'known good' 
> list
> 5. Ultimately, even FS 'plugins' could be created although IPL would be 
> required to load these. 
> 6. Code for Kernel, Interface and 'plugins' would still be GPL. This would not 
> prevent the 'tainted' system idea. 

So "drivers" are a third class citizen?  They don't need to be under the
GPL for some reason?

> Expected Outcomes
> ------------------------
> 
> 1. Make Linux easier to use

How would this help this?  The kernel would get bigger somehow, right?

> 2. The ability to replace even very core Kernel components without a restart.

We can do that today with modules.

> 3. Allow faulty 'plugins' to be fixed/replaced in isolation. No other system 
> impact.

How are you going to isolate parts of the kernel from itself?

> 4. 'Plugins' could create their own interfaces as load time. This would remove 
> the need to pre-populate /dev. 

What?

Ok, I'm just giving up now.

But remember, patches are always welcome.  Please post your code to
implement this system if you come up with some.

Good luck,

greg k-h

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Richard B. Johnson]

On Wed, 3 Sep 2003, James Clark wrote:

> Following my initial post yesterday please find attached my proposal for a
> binary 'plugin' interface:
>

But we already have a well-defined modular interface. It allows
(driver) modules to not only be installed into a running kernel,
but also to be removed from a running kernel. This makes driver
development quite easy compared to others where one needs to
re-boot to make the .... "mouse movement detected, reboot to make
the changes take effect.."

The well-defined interface for the modules is "struct file_operations".
It corresponds to the Unix I/O model for devices. This interface
is not broken and is, therefore, not necessary to be changed.

There is a Microsoft NT model where software is put somewhere and
its functions are 'registered' with the kernel. This is an abortion
designed totally to hide the inner workings of the kernel. Since
our source-code is visible, we don't need layers of abstraction
to high it from developers.

So, to your proposal, no thanks. Also, there is no way that
I will __ever__ allow software provided by others to execute
within the kernel of anything I am responsible for, unless that
software's source-code is available for inspection.

The recent problems with the Microsoft worms should certainly
convince any sane person that we can't allow some hidden software
to run our industry no matter how well we recognize the name.

There are two main reasons for using Linux in embedded systems:

(1) I know what's in there.
(2) I can make hardware drivers without having to reboot the target.

Before Linux, we had to roll our own Operating Systems. And, yes,
we made ARTOS (Analogic Realtime Operating System). We created
a whole 'C' runtime library environment. It was a bitch. We don't
need to do this anymore because there is Linux. If you, or any
other well-wisher ever succeeds in converting Linux to a binary,
no-source system, then Linux will get forked and your interface
will get fu*k^M^Mmorked also.

Cheers,
Dick Johnson
Penguin : Linux version 2.4.22 on an i686 machine (794.73 BogoMips).
            Note 96.31% of all statistics are fiction.

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Patrick Mochel]

> Following my initial post yesterday please find attached my proposal for a 
> binary 'plugin' interface:

Whoa, whoa, whoa! Hang on there, cowboy. 

It's always easier to observe the behavior of a system from the outside
and make suggestions from that objective point of view (especially when
you have experience in another similar system). However, change can
ultimately only come from within.

We've always been open to outside suggestions, but they're really only 
useful when they contain implementation details (usualy in the form of 
working code) that build on top of our existing system. 

Linux is an evolutionary process. While it's tempting to project the 
behavior or ideals of another similar system on to it -- simply because 
you can -- it really doesn't work. We evolve and mutate and change. And, 
that's where we feel we are really superior. 

I've not read your proposal -- nor do I intend to -- even as maintainer of
the new driver model. I do however encourage you to pick up a copy of 
"Linux Device Drivers", read it, and start contributing to the development 
process. After getting a feel for how we work, reconsider your proposal, 
and submit some patches to implement your ideas. 

Until then, I'm not interested in discussing license issues or binary
modules, so please remove me from the CC list.

Thanks,


	Pat

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Gábor Lénárt]

On Wed, Sep 03, 2003 at 06:53:01PM +0100, James Clark wrote:
> Following my initial post yesterday please find attached my proposal for a 
> binary 'plugin' interface:

Plugin? Ugly word. Let's say modules instead. And modules ARE implemented
now with proper interface, so what are we talking about?
 
> This is not an attempt to have a Microkernel, or any move away from GNU/OSS 
> software. I believe that sometimes the ultimate goals of stability and 

I can't follow. So you assumed that microkernel can't be implemented in
open source environment? Let's check out Hurd (btw, afaik Hurd is a collection
of 'server processes' running on the top of the Mach microkernel).

> portability get lost in the debate on OSS and desire to allow anyone to 
> contribute. It is worth remembering that for every Kernel hacker there must 

? Anyone can write a kernel module even now, I don't see the problem
here.

> be 1000's of plain users. I believe this proposal would lead to better 
> software and more people using it.
> 
> Proposal
> -----------
> 1. Implement binary kernel 'plugin' interface

binary? You can't. For being precise: you can't do it with keeping
performance in mind. Newer major kernel releases use more or less
different kind of structures, algorithms, etc. There are SMP systems
and UP and so on. For maximum performance, kernel module SHOULD be
compiled from source with target configuration keeping in mind and
matching with kernel's configuration of course.

Sure, you can create a 'glue' layer between the actual kernel module
API and something you're calling 'binary interface', but it will be
a performance bottleneck, also this would make kernel larger in memory.
You will end at a situation where win NT also sucks: several abstracion
layers should be implemented which wastes performance just "translating"
things between software and/or hardware resources, while it can be possible
to do this directly. Sure, direct method breaks compatibility, eg you can't
load easily a 2.0.x kernel modules into 2.6.0-testX kernel, but exactly
this is the why where you're not limited by the bad meaning of compatibility
just for wasting several percent of your resources to keep everything 
working with VERY obsolated binary-only drivers ...

> 2. Over time remove most existing kernel 'drivers' to use new interface - This 
> is NOT a Microkernel.

I can't understand, let's say:

A) "Over time remove most existing kernel 'drivers' to use new interface"
B) "This is NOT a Microkernel"

I can't see any logical relationship between A and B, so I don't understand
this sentence at all.

> 3. Design 'plugin' interface to be extensible as possible and then rarely 
> remove support from interface. Extending interface is fine but should be done 
> in a considered way to avoid interface bloat. Suggest interface supports 
> dependant 'plugins'

So if someone 'invents' a brand-new algorithm for Linux you SHOULD keep
the binary interface the same, where you can gain (let's say) 1000% of
performance with reconstructing the module interface as well? Ehhh. Or
you should keep up an 'abstraction layer' to translate between 'good-old-braken'
API and the real "internal" kernel API? It would be HORRIBLE, SLOW, HUGE,
and UNSTABLE. This reminds me m$ win at the first sight ;-)

> 4. Allow 'plugins' to be bypassed at boot - perhaps a minimal 'known good' 
> list
> 5. Ultimately, even FS 'plugins' could be created although IPL would be 
> required to load these. 
> 6. Code for Kernel, Interface and 'plugins' would still be GPL. This would not 
> prevent the 'tainted' system idea. 

? Please, say only ONE advantage of this binary plugins thing then!
Because I don't see even ONE, just disadvantages, I've described.

> 
> Expected Outcomes
> ------------------------
> 
> 1. Make Linux easier to use
> 2. The ability to replace even very core Kernel components without a restart.

Modules. It's only a question to modularize the current kernel even more,
not something 'we need some very different model'. But this question would
lead to the fact that you can't modularize at this 'brutal' way, since
between versions of Linux, core changes can be done as well. So to do this
either you need to a very-complex-and-almost-imposible-to-implement
'translator' which would save internal stage of the whole system, and
loads it to the probably very different new data structures, or linux
should keep binary compatibility between versions with slow, huge, and
unstable 'glue' layers, hiding internal things from other parts.

> 3. Allow faulty 'plugins' to be fixed/replaced in isolation. No other system 
> impact.
> 4. 'Plugins' could create their own interfaces as load time. This would remove 
> the need to pre-populate /dev. 

??

> 5. Remove need for joe soap user to often recompile Kernel.

User should not recompile the kernel even now. I have several 'windows-style'
friends using Linux without even know the meaning of 'kernel' at all.

> 6. Remove link between specific module versions and Kernel versions.

You CAN'T do this, I've described. Sure, you can, but there would be
only disadvantages.

> 7. Reduce need for major Kernel releases. Allow effort to concentrate on 
> improving Kernel not maintaining ever increasing Kernel source that includes 
> support for the 'Kitchen Sink'
> 8. Make core Kernel more stable. Less releases and less changes mean less 
> bugs. It would be easy to identify offending 'plugin' by simply starting up 
> the Kernel with it disabled.
> 9. Remove need for modules to be maintained in sync with each Kernel thus 
> freeing 'module' developers to add improved features or work on new projects.

So again, you can't gain the maximum of your hardware because the complexity
of that 'binary API' would be something like a low bandwidth network
between two supercomputers, eg with creating a single on-time binary API,
you can't gain the maxmimum of new invents later without breaking it.


 
> James
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 
- Gábor (larta'H)

Re: Driver Model 2 Proposal - Linux Kernel Performance v Usability

[Christoph Hellwig]

On Wed, Sep 03, 2003 at 06:53:01PM +0100, James Clark wrote:
> Following my initial post yesterday please find attached my proposal for a 
> binary 'plugin' interface:
> 
> This is not an attempt to have a Microkernel, or any move away from GNU/OSS 
> software. I believe that sometimes the ultimate goals of stability and 
> portability get lost in the debate on OSS and desire to allow anyone to 
> contribute. It is worth remembering that for every Kernel hacker there must 
> be 1000's of plain users. I believe this proposal would lead to better 
> software and more people using it.

Looks like you're looking for UDI.  Most people on this list don't like
it for various reasons (me included) but you seem to like it.  Since
SCO was the driving factor development probably has slowed down even
more, go and help them!

Don't expect it to ever be merged though..

	http://projectudi.sourceforge.net/