Re: Linux drivers management

Re: Linux drivers management

[linux]

> Is there any work in Linux undergoing to separate Linux drivers and the 
> the main kernel, and manage drivers using a package management system 
> that only manages kernel drivers and modules? If this can be done, the 
> kernel maintenance can be simple, and will end-up with a more stable 
> (less frequent changed) kernel API for drivers, also make every 
> developers of drivers happy.

Not very seriously.  Kernel developers really like the ability to change
every user of a kernel programming interface within a single source tree.
Breaking it up would make it harder to change the device driver interface
when necessary.  (It's already hard enough; nobody does it for fun.)

Also, a hardware manufacturer looking for a "stable API" is often
really looking for a stable *binary* interface because they want to
ship binary-only drivers.

The Linux developers are quite opposed to that, for a variety of excellent
reasons I won't bother enumerating.  Linus has said he'll (grudgingly)
allow it, but won't lift a finger to help.  Linux development sailed
away from the idea of a stable binary interface years ago, and isn't
looking back.

Re: Linux drivers management

[Eric W. Biederman]

linux@horizon.com writes:

> The Linux developers are quite opposed to that, for a variety of excellent
> reasons I won't bother enumerating.  Linus has said he'll (grudgingly)
> allow it, but won't lift a finger to help.  Linux development sailed
> away from the idea of a stable binary interface years ago, and isn't
> looking back.

Almost true.  There is a stable binary interface to user space,
and work is done to maintain that interface.

I just thought I would mention it because too frequently people
get the policy on the in-kernel api and the ABI to user space
confused.

In general the user space ABI is only appended to.

Eric

Re: Linux drivers management

[David Chow]

Dear all,

First of all, thanks for giving me lots of wonderful feedbacks, I read 
both pros and cons, support and against msgs. And you are right, I am 
serious person about this matter, not joking. Because I do this for my 
bread and butter, not just for fun.

Before I continue this discussion, I would really want to clarify who am 
I before get discriminated by end-users and developers, because I am both.

I am a full-time (M$_less) Linux user (many years...., Sharp Zaurus PDA, 
Linux on laptop, desktop, server, storage and datacenter) with zero 
windows experience for the last 5 years (can't remember, my last windows 
desktop was on NT 4.0/Win98). I am an engineer, I wrote many file 
systems (cogofs http://www.shaolinmicro.com/product/cogofs/, unionfs, 
.....more), cluster infrastructure (used in ShaoLin InfiniCluster 
http://www.shaolinmicro.com/product/icluster/technology.php), data 
replication (used in Shaolin Volume Replicator), load balancing, iSCSI 
storage drivers.....  . I am also a system architect/engineering 
manager, architect core Linux system software from scratch, desktop, 
network computing, servers, network related projects. I am also a R&D 
team manager, hiring, training, manage engineers .

In fact, this is not to talk about myself, but my full-time Linux 
experience tells me Linux development is walking away with end-uesrs and 
commercial developers that try to work with Linux. It sounds too scary 
by now, with over 40MB tarred gzip kernel source. With personal efforts 
and company resources, we maintain more than hundreds (probably 
thousand) of Linux kernel headers and sources in our lab for building 
off-the-shelf supported binary drivers with multiple hardware archs 
(x86, ppc...). Because of this, who is going to pay for this? It doesn't 
make any sense for anyone who want to maintain and write a driver for 
Linux have to pay this price.

My summary from the responses from lkml collected as follows. Please 
feel free to make correction if I am wrong, or comment if you want, I 
will minimize to say about subjective comment to others comment in an 
open area though.

Community Developers and Maintainers:
- Look at the matter on community development process, programming
- Chase for performance, optimization in source level, even though it is 
difficult to maintain, who cares?
- Want freedom, change at will (with supported arguments, but who cares?) .
- My feels like it doesn't consider any other forms of development and 
respect to traditional software engineering or QA process. Because a 
stable API is first needed for teamwork collaborative development.
- Willing to maintain and develop drivers for free, even though they 
don't work for the hardware vendor.
- We will follow the convention who make changes to the API will have to 
patch all the mess in the kernel source, even though there are 3,714,234 
hardware peripheral drivers in the kernel in year 2012, I am happy to do 
that :) . Because I want to make change and following the convention. 
(how much time to make change or test?)

Technical End-Users:
- Want to compile the drivers from source
- Enjoy building their own kernel, apply patches (patch and make, it 
works! thats cool....)
- I don't mind to search for drivers and do it myself, because it was 
fun to make something work with my effort :) .
- I don't mind to upgrade my OS because of a missing driver or needed 
for new fucntionality. Even my application breaks, down time is not 
important to my system because it s a sytsem for fun.

Non-technical Users:
- Want the system to have drivers pre-built, so that they don't have to 
go through a compilation or patching process. Its a waste of time for 
them (waste of time for me too)
- Why I have to search the drivers? Isn't is suppose to be included in 
the OS? Or if not included in the OS, it should be included in a driver 
disk (CD/DVD/floppy or whatever medium or download) .
- Why I have to upgrade the complete OS if only one driver is missing? I 
want to stay with Redhat-9 , my PHP runs great.
- There is no "Linux support" labels on most the hardware out there, 
should I risk my money, buy it and try out? Oh, full refund of item is 
not allowed . Then, don't bother ...

Commercial developers:
- Want a stable API so that drivers can be maintained with ease. Because 
we don't just work with Linux, we want to focus on our driver 
development, not chasing the API changes, versions by versions, vendors 
by vendors. Sometimes there are even vendor specific changes, its a 
waste of time.
- If I have to make binary drivers, I have to maintain all kernel 
sources and headers, compilers to make sure my drivers will be built 
correctly without problem. Of risk to change symbols in the binaries and 
hope it works!
- Where is the latest up-to-date documentation of the kernel API? 
/Documentation only partially describe what I need, its version 
specific, sometimes out-of-date, where the hell is that? Let's google it 
in amazon.com, "Linux driver books", No good again.... Its crap, all not 
up-to-date!
- Lets get on to it, read all docs and sample sources... mmm... My 
driver seems working now.. Lets compile it and distribute it. Users: 
have you got a driver for Redhat 9 2.4.18 kernel? Answer: No, it doesn't 
work, because I write my driver on 2.6.15, you may to DIY. User 
response: I want a refund, because you said your hardware has Linux 
support, but its a false statement.
- Just leave Linux, who cares, it doens't make sense to us. Because it 
doesn't make sense to go through all these problems to say "Linux 
supported hardware", user will get refund the product if we say this on 
the box on day one.
- Maybe we have another way to do that, submit the driver to the 
community and hope it to include it in the latest kernel source. 
Wait.... but what about support for Redhat 9 and SuSe 8.2?
- We are happy to maintain our own drivers, because we know better about 
our hardware. We are paid to do so, we also have quality assurance 
process with formal test tools and equipment. Don't think the community 
can do a better core than us.
- It just doesn't work for us. No more Linux driver Cd's, it will not 
happen .

My comments:
- Freedom? Someone tell me to shut-up . Some people define freedom using 
their own way, not even using mailling list for discussion and make 
suggetions or even define questions as "stupid", I will not say that to 
anyone. Its rude.
- Wake up! Why would the maintainers bother to maintain the drivers if 
the driver development work is now back to the hardware vendor, like 
drivers for other platform did? I think someone mis-understood the whole 
idea is to "GET RID OF DRIVER MAINTENANCE", belive it or not, it belongs 
to the vendor, not here. If the driver releases as GPL, you can still 
make your own changes, but it doesn't have to be in main source tree.
- You plug-in the hardware, it worked! Because many people behind the 
scene has done a lot of work. My purpose of raising this question is 
trying to help both users and developers, and try to make more hardware 
that behaves "plug-n-play" .
- What is the goal of Linux developers? Just for fun? Or you want Linux 
to get more popular? Users want their system to get supported with 
latest drivers, not to compile and build to latest kernel. Or not to 
upgrade their Linux distro every week or month. I don't use 2.6.15 nor 
happy downloading 40Mb targged gzip kernel source and knowing how to 
"make" it.
- Linux will not sail to major desktop unless a decent DDK (driver 
development kit) exists. There is a stable ABI on the user space, but 
the hardware has to "get worked" before anything in user space happens. 
I decide to sell my USB wireless-G adapter because I don't have a driver 
for it, neither Linksys did. I can only choose to get rid of Linux, but 
can't, so just sell it. For others, why don't they simply choose another 
supported OS?
- /Documentation/stable_api_nonsense.txt is only a document totally 
written by a programmer sense, its nothing about people who don't want 
to compile the drivers, and has assumed drivers should be maintained by 
the community. But strictly speaking, it shouldn't. Please refer to the 
process of making a driver from a manufacturers point of view and 
consider user using old OS'es which don't want to upgrade.

Final comment: There is no right or wrong, stupid or smart, it depends 
where you stands and where you want Linux to go. I am very clear myself 
is to get Linux promoted to public sectors (where I belive 99% users are 
non-technical), easy for developers (I believe everyone wants 
easy-way-out) and easy for the community (I belive you people like 
innovation, new ideas, rather thatn spend your time to work/maintain 
drivers which this work should belongs to the original vendor). If you 
think I have no contribution and stupid, that's up to you (who cares? 
Linux has been working like this in day-one that I first compile and run 
it). But my work has already beyond programming, because making patches 
for Linux doesn't make any sense to me, especially when porting drivers 
that I can't even tell what they are. My mood of patching the kernel 
goes away when today's Linux kernel targged gzip source gets to over 
40MB .. I have more important things to do. Its enough for me by now.... 
Sure its not going to change, maybe but not in a year or two, but 
freedom of speech exists, right?

regards,
David Chow

Re: Linux drivers management

[Kyle Moffett]

On Feb 07, 2006, at 14:45, David Chow wrote:
> Before I continue this discussion, I would really want to clarify  
> who am I before get discriminated by end-users and developers,  
> because I am both.
>
> [offtopic babble about credentials]

We do not discriminate based on "end-user" or "developer", we  
discriminate based on productivity.  So far this thread has been  
extremely counterproductive and wasted a lot of my bandwidth and  
time.  As a result, I am now discriminating against you for wasting  
my time.  Welcome to my killfile.  (I still felt the need to point  
out your grievous logical errors, but don't bother replying because  
this is the last time I'm going to bother wasting time on this thread)

> [junk about commercial development models]

We do not care about your snazzy dev-model ideas, we have one that  
works for us.  We do not care about making things easier for  
commercial out-of-tree drivers, _end_ _of_ _story_.  Any arguments  
about that issue are just offtopic flames.
> Why would the maintainers bother to maintain the drivers if the  
> driver development work is now back to the hardware vendor, like  
> drivers for other platform did? I think someone mis-understood the  
> whole idea is to "GET RID OF DRIVER MAINTENANCE", belive it or not,  
> it belongs to the vendor, not here. If the driver releases as GPL,  
> you can still make your own changes, but it doesn't have to be in  
> main source tree.

WRONG.  Driver maintenance is a 2-part effort.  The Linux kernel API  
is *not* stable for a lot of good reasons, and therefore the drivers  
must be in-tree to make it possible to fix drivers when we change the  
API.  Hardware companies are _expected_ to be good citizens and  
maintain their own drivers, fix bugs, etc.  If your driver sucks,  
nobody will buy your hardware to use on linux.

> Linux will not sail to major desktop unless a decent DDK (driver  
> development kit) exists.

This is wrong.  There are a lot of companies that make great server  
hardware out there whose drivers are in the stock kernel, and by your  
argument "Linux will not sail to major servers unless a decent DDK  
exists", which is blatantly false.

> /Documentation/stable_api_nonsense.txt is only a document totally  
> written by a programmer sense

Absolutely, and from the programmer point of view, that's all that  
matters.

> its nothing about people who don't want to compile the drivers

This is the job of a distro

> and has assumed drivers should be maintained by the community.

Community includes the people making the hardware

> But strictly speaking, it shouldn't. Please refer to the process of  
> making a driver from a manufacturers point of view and consider  
> user using old OS'es which don't want to upgrade.

You don't want to upgrade, you don't get new hardware support, simple  
as that.  Upgrading my Debian testing between 2.6 versions has been  
really painless despite massive internal changes and restructuring,  
and Debian isn't really even a user-friendly distro.

> but freedom of speech exists, right?

As does my freedom to ignore you. Plonk.

Cheers,
Kyle Moffett

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCM/CS/IT/E/U d- s++: a18 C++++>$ ULBX*++++(+++)>$ P++++(+++)>$ L++++ 
(+++)>$ !E- W+++(++) N+++(++) o? K? w--- O? M++ V? PS+() PE+(-) Y+ PGP 
+ t+(+++) 5 X R? !tv-(--) b++++(++) DI+(++) D+++ G e>++++$ h*(+)>++$ r 
%(--)  !y?-(--)
------END GEEK CODE BLOCK------

Re: Linux drivers management

[Theodore Ts'o]

On Wed, Feb 08, 2006 at 03:45:47AM +0800, David Chow wrote:
> - What is the goal of Linux developers? Just for fun? Or you want Linux 
> to get more popular? Users want their system to get supported with 
> latest drivers, not to compile and build to latest kernel. Or not to 
> upgrade their Linux distro every week or month. I don't use 2.6.15 nor 
> happy downloading 40Mb targged gzip kernel source and knowing how to 
> "make" it.

Every Linux developer has their own goals, of course, but for most of
them it is about making the best possible Linux kernel that is
technically possible.  If they have working drivers for their system,
they may not necessarily care about some company's hardware unless,
(a) it impacts them personally, (b) they are paid or employed to worry
about it, or (c) lots of end-users are sending complaining/sending
hate-mail about it.  

(In some cases, end-users send hate mail to the Linux kernel
developers when some idiot company's binary driver modules is buggy
and corrupts the kernel in hard-to-debug ways; one particular video
driver company is especially guilty here, and is viewed by some as
being directly responsible for the tainted kernel flags.)

The assumption by many developers is that if we concetrate on making
Linux as good as possible, it will eventually get popular enough that
hardware vendors will feel a commercial incentive to cooperate with
our way of doing things.  Obviously, this in practice things don't
always work that way --- the Sony Betamax is story is one where
technical excellence doesn't always win out.  However, at least in the
server space, compromising hasn't obviously been a bad strategy, with
many SCSI and FC controller manufacturers deciding on their own to
work with the Linux kernel development community.  (Sometimes with
some help from major system vendors who write in a requirement for a
mainline device driver into the sourcing contracts for said
controllers, but nevertheless, it shows that this stance is not
obviously a bad strategy for Linux kernel developers, at least in the
server space.)

David, you may find this frustrating, and at least in the Deskstop
space, it's likely that your company hasn't seen sourcing contracts
yet where a mainline acceptable device driver is a requirement for
some major system vendor, like Dell, Gateway, HP, etc. to decide to
use your products.  I suspect that if this _was_ the case, your
company would in fact dedicate the full-time engineer necessary to
make a device driver which could be integrated into the mainstream
kernel sources and then could be backported to older distributions.
But if you did, I think it is certainly doable.

But at that point it stops being a technical question of "is it
possible" and moves to an economic question of "are we willing to fund
a full-time engineer to provide support for our hardware under Linux"
and "how popular does the Linux desktop have to be before a system
vendor will feel obliged to put pressure on their downstream suppliers
to provide the necessary driver support"?  And as such, LKML will
probably not be a very useful place to have that discussion.

Regards,

						- Ted

Re: Linux drivers management

[David Chow]

>Every Linux developer has their own goals, of course, but for most of
>them it is about making the best possible Linux kernel that is
>technically possible.  If they have working drivers for their system,
>they may not necessarily care about some company's hardware unless,
>(a) it impacts them personally, (b) they are paid or employed to worry
>about it, or (c) lots of end-users are sending complaining/sending
>hate-mail about it. 
>  
That's expected. IS there a composition statistics about the LKML? I 
guess near 100% are technical people here, including me.
>(In some cases, end-users send hate mail to the Linux kernel
>developers when some idiot company's binary driver modules is buggy
>and corrupts the kernel in hard-to-debug ways; one particular video
>driver company is especially guilty here, and is viewed by some as
>being directly responsible for the tainted kernel flags.)
>
>The assumption by many developers is that if we concetrate on making
>Linux as good as possible, it will eventually get popular enough that
>hardware vendors will feel a commercial incentive to cooperate with
>our way of doing things.  Obviously, this in practice things don't
>always work that way --- the Sony Betamax is story is one where
>technical excellence doesn't always win out.  However, at least in the
>server space, compromising hasn't obviously been a bad strategy, with
>many SCSI and FC controller manufacturers deciding on their own to
>work with the Linux kernel development community.  (Sometimes with
>some help from major system vendors who write in a requirement for a
>mainline device driver into the sourcing contracts for said
>controllers, but nevertheless, it shows that this stance is not
>obviously a bad strategy for Linux kernel developers, at least in the
>server space.)
>
>David, you may find this frustrating, and at least in the Deskstop
>space, it's likely that your company hasn't seen sourcing contracts
>yet where a mainline acceptable device driver is a requirement for
>some major system vendor, like Dell, Gateway, HP, etc. to decide to
>use your products.  I suspect that if this _was_ the case, your
>  
No, I never had drivers problems . Because we have our own stable 
partial_kernel_API to bare this problem and kept all supported kernel 
sources and headers maintained.
>company would in fact dedicate the full-time engineer necessary to
>make a device driver which could be integrated into the mainstream
>kernel sources and then could be backported to older distributions.
>But if you did, I think it is certainly doable.
>  
Yes it worked for us. But what about others? I don't think everyone that 
want to support Linux have to do that. We are different, because we only 
support Linux, so we dare to do that. Other companies have to do 
Windows, Unix and possibly other OS. This way don't seems feasible for them.
Back-port?
>But at that point it stops being a technical question of "is it
>possible" and moves to an economic question of "are we willing to fund
>a full-time engineer to provide support for our hardware under Linux"
>and "how popular does the Linux desktop have to be before a system
>vendor will feel obliged to put pressure on their downstream suppliers
>to provide the necessary driver support"?  And as such, LKML will
>probably not be a very useful place to have that discussion.
>  
I have no expectation the LKML will agree to my point . Because 99% of 
the LKML are likely technical users and community developers. As said, 
they only care about programming drivers in the kernel source. Freedom 
of change is cool and fun for them.


regards,
David Chow

Re: Linux drivers management

[Theodore Ts'o]

On Wed, Feb 08, 2006 at 08:52:15AM +0800, David Chow wrote:
> I have no expectation the LKML will agree to my point . Because 99% of 
> the LKML are likely technical users and community developers. As said, 
> they only care about programming drivers in the kernel source. Freedom 
> of change is cool and fun for them.

Actually, most of the developers work for some a distributions or a
system vendor.  I for example happen to work for IBM's Linux
Technology Center.  In that role, I do worry about the device drivers
for the hardware devices that we sell to our customers.  However, I
also am a community developer, and with that hat on I care about Linux
being the best OS it can be technically.  

I will say that my experience has been that binary kernel modules can
easily turn into a disaster for our customers.  It's a major issue
when a customer needs to install an errata kernel issued by one of our
Linux Distribution Partners for stability or security reasons, and
they are using a propietary binary kernel module from some third party
storage vendor which hasn't yet updated their proprietary binary
driver for that errata kernel.

Or there was the time that positively warmed my heart when I spent
several very late nights trying to debug a situation for a very high
profile, important customer trying to use a Samba file server running
on IBM hardware being integrated by IBM Global Services, and the
system kept on falling over.  It turned out the problem was caused by
a memory leak in a propietary, binary-only kernel module from an
commercial anti-virus product that shall remain nameless.   

So I am firm believer in giving our customers access to source code to
all kernel code, not because of any deep desire to "programming
drivers in kernel source", or because of any "information wants to be
free" religion --- but because it's the best way to keep our
customer's systems running reliably and nearly problem-free --- and
when there is a problem, I know that we have whatever is necessary to
make their problem Go Away.

Yes, I'm aware of the traditional arguments that a stable device
driver API would solve all of these problems.  Well.... at least the
first problem; incompetently written propietary kernel modules filled
with memory leaks and wild pointer dereferences and race conditions
aren't solved by standardized driver API's; the only solution is
source access so we can fix the idiotically written modules.  But the
reality is the way the Linux kernel is developed, a stable driver API
would never work.

						- Ted

P.S.  Obligatory disclaimer: These are my own opinions, and not
necessarily those of my employer.

Re: Linux drivers management

[Bernd Petrovitsch]

On Wed, 2006-02-08 at 08:52 +0800, David Chow wrote:
[...]
> >(In some cases, end-users send hate mail to the Linux kernel
> >developers when some idiot company's binary driver modules is buggy
> >and corrupts the kernel in hard-to-debug ways; one particular video
> >driver company is especially guilty here, and is viewed by some as
> >being directly responsible for the tainted kernel flags.)
> >
> >The assumption by many developers is that if we concetrate on making
> >Linux as good as possible, it will eventually get popular enough that
> >hardware vendors will feel a commercial incentive to cooperate with
> >our way of doing things.  Obviously, this in practice things don't
> >always work that way --- the Sony Betamax is story is one where
> >technical excellence doesn't always win out.  However, at least in the

But only because the company behind BetaMax (and BetaMax was only second
best behind Philips Video2000 AFAIR) had not enough money and/or
motivation to stay long enough to make the technical better solution
also successful.
IOW you won't get the perspective right if you view from the commercial
old-ecenomy world only (as you do up to now!)

> >server space, compromising hasn't obviously been a bad strategy, with
> >many SCSI and FC controller manufacturers deciding on their own to
> >work with the Linux kernel development community.  (Sometimes with
> >some help from major system vendors who write in a requirement for a
                  ^^^^^^^^^^^^^^^^^^^^
Do you knwo the contracts, agreements and result of meetings of major of
major system vendors with the sales army of big OS corporations in the
desktop area?
Probably not.

> >mainline device driver into the sourcing contracts for said
> >controllers, but nevertheless, it shows that this stance is not
> >obviously a bad strategy for Linux kernel developers, at least in the
> >server space.)

Perhaps you should ask them (and I mean "ask the folks to pushed and
took the decision and not spokespersons or marketing and sales staff")
"why on earth?".

[...]
> Yes it worked for us. But what about others? I don't think everyone that 
> want to support Linux have to do that. We are different, because we only 
> support Linux, so we dare to do that. Other companies have to do 
> Windows, Unix and possibly other OS. This way don't seems feasible for them.

But it is *their* commercial decision if they play the Linux market (as
long as it exists) and/or if they play the proprietory market (as long
as it exists). As well as it is MSFT decision to play exclusively in the
propriatory market, they and you have to live with it.
Of course we all want the best of all worlds - free software,
out-of-the-box working toolchains and stable APIs through all of them to
minimize my own work. IMHO this is another "choose any 2 of the 3"
question.

> Back-port?

If you want a commercial answer: Either it pays off or not. It is
*their* decision.

> >But at that point it stops being a technical question of "is it
> >possible" and moves to an economic question of "are we willing to fund

Yes, because the free software world generally gives a broad "yes, it is
technically possible" (whereas the propriatory world makes of this the
key selling point).

> >a full-time engineer to provide support for our hardware under Linux"
> >and "how popular does the Linux desktop have to be before a system
> >vendor will feel obliged to put pressure on their downstream suppliers
> >to provide the necessary driver support"?  And as such, LKML will
> >probably not be a very useful place to have that discussion.

Probably wrong. It is completely up to the hardware vendor to decide if
-) they just want to sell their hardware and release specs so that
   everyone can write (and fix) a (free) driver.
-) also have reasons to maintain the drivers themselves over time.
-) just out-source the development and maintenance of a driver.
-) just out-source the development of a driver for the current kernel
   (to you, me, someone), release the source under GPL, throw it into
   the kernel and copnsider (and sell!) it as "maintained".
-) divide the driver in one OS-independent lower part and a OS-dependent
   upper part (whareas the latter is surely under the GPL and with the
   former you have to look at each case) to "hide hardware details"
   for whatever reason.
-) or whatever partial "solutions" (and combinations therof over time)
   come to the mind of some deciders.
*The hardware vendor* (and not me and IMHO not LMKL) has to decide which
fits most in his business plans. *The hardware vendor* makes the money
with after all.

> I have no expectation the LKML will agree to my point . Because 99% of 
> the LKML are likely technical users and community developers. As said, 
This is probably correct:^^^^^^^^^^^^^^^
This is plain simply wrong:               ^^^^^^^^^^^^^^^^^^^^
There are for sure *far* more then 1% who make living with
"Linux" (technically wise).

> they only care about programming drivers in the kernel source. Freedom 
> of change is cool and fun for them.

As long as you want to see (and/or argue) exactly the historically grown
propriatory business model, it certainly doesn't work out.

	Bernd
-- 
Firmix Software GmbH                   http://www.firmix.at/
mobil: +43 664 4416156                 fax: +43 1 7890849-55
          Embedded Linux Development and Services

Re: Linux drivers management

[Lee Revell]

On Tue, 2006-02-07 at 17:15 -0500, Theodore Ts'o wrote:
> (In some cases, end-users send hate mail to the Linux kernel
> developers when some idiot company's binary driver modules is buggy
> and corrupts the kernel in hard-to-debug ways; one particular video
> driver company is especially guilty here, and is viewed by some as
> being directly responsible for the tainted kernel flags.) 

Wouldn't the tainted kernel flags be necessary even if there had never
been a single bug in any binary driver, simply because there's still no
reasonable way to debug a kernel with binary drivers loaded?

Lee

Re: Linux drivers management

[Alan Cox]

On Mer, 2006-02-08 at 03:45 +0800, David Chow wrote:
> Community Developers and Maintainers:
> - Look at the matter on community development process, programming
> - Chase for performance, optimization in source level, even though it is 
> difficult to maintain, who cares?

I've done real studies on this. The majority of the kernel interfaces
favour simplicity and independance.

> - Willing to maintain and develop drivers for free, even though they 
> don't work for the hardware vendor.

Most of our drivers are maintained by people with economic incentives to
maintain them. Economic incentives are not always cash and employment.

> - We will follow the convention who make changes to the API will have to 
> patch all the mess in the kernel source, even though there are 3,714,234 
> hardware peripheral drivers in the kernel in year 2012, I am happy to do 
> that :) . Because I want to make change and following the convention. 
> (how much time to make change or test?)

I'd beg to differ. The amount of hardware interfaces in a system that
are non standard has been dropping like a stone. Software is *expensive*
and it is getting cheaper and cheaper for commodity products to adopt a
commodity API, especially in the open source world where pure software
pricing scams (eg extra cost for flipping the 'raid enable' bit on a
controller) don't work.

The recent directions are pretty clear
	IDE -> Various interfaces -> AHCI
	A billion periphals -> USB (EHCI/OHCI/UHCI) + Class drivers
	SATA -> AHCI
	Sound -> Intel i810 clones/AC97
	USB imaging random drivers -> USB video classes
	Scanners -> USB video/image classes
	A billion camera interfaces -> USB storage
	A load of MP3 player interfaces -> USB storage

Its getting harder and harder to justify non-standard APIs except in a
few areas like infinibong and 3D graphics where they can make a product
genuinely better. Even RAID cards are drifting inexorably to either
extinction or emulating standard interfaces, and no doubt will
eventually end up AHCI.

> the community. But strictly speaking, it shouldn't. Please refer to the 
> process of making a driver from a manufacturers point of view and 
> consider user using old OS'es which don't want to upgrade.

But those people have a stable API - usually RHEL3, RHEL4, SuSE
Enterprise. 

> Sure its not going to change, maybe but not in a year or two, but 
> freedom of speech exists, right?

Time will tell. People said the same when Linux got so big it wouldn't
fit on one floppy disk. I can't prove you are wrong but my suspicion is
that economic incentives and pressures will mould the development
process over time according to the problems it faces.

Alan

Re: Linux drivers management

[Denis Vlasenko]

On Tuesday 07 February 2006 21:45, David Chow wrote:
> Non-technical Users:
> - Want the system to have drivers pre-built, so that they don't have to 
> go through a compilation or patching process. Its a waste of time for 
> them (waste of time for me too)
> - Why I have to search the drivers? Isn't is suppose to be included in 
> the OS? Or if not included in the OS, it should be included in a driver 
> disk (CD/DVD/floppy or whatever medium or download) .
> - Why I have to upgrade the complete OS if only one driver is missing? I 
> want to stay with Redhat-9 , my PHP runs great.

Then why MS has that auto-update service of theirs?

> - There is no "Linux support" labels on most the hardware out there, 
> should I risk my money, buy it and try out? Oh, full refund of item is 
> not allowed . Then, don't bother ...

Because hardware verdors act stupid and many of them still do
not write open-source drivers (or at least provide adequate docs).

> Commercial developers:
> - Want a stable API so that drivers can be maintained with ease. Because 
> we don't just work with Linux, we want to focus on our driver 
> development, not chasing the API changes, versions by versions, vendors 
> by vendors. Sometimes there are even vendor specific changes, its a 
> waste of time.
> - If I have to make binary drivers, I have to maintain all kernel 
> sources and headers, compilers to make sure my drivers will be built 
> correctly without problem. Of risk to change symbols in the binaries and 
> hope it works!

IOW: "we want to hijack millions of lines of Linux source and won't
contribute back our driver(s). Why do you kernel guys make that hard?"

Because we don't like what you're doing.

> - Where is the latest up-to-date documentation of the kernel API? 
> /Documentation only partially describe what I need, its version 
> specific, sometimes out-of-date, where the hell is that? Let's google it 
> in amazon.com, "Linux driver books", No good again.... Its crap, all not 
> up-to-date!

The source is ultimate doc. You never ever will get such a complete doc
for any commercial OS. It even documents all bugs! ;)

> - Lets get on to it, read all docs and sample sources... mmm... My 
> driver seems working now.. Lets compile it and distribute it. Users: 

Wrong. You should do: "Let's submit it for inclusion in mainline".
If you don't want to, it's your problem, not ours.

> have you got a driver for Redhat 9 2.4.18 kernel? Answer: No, it doesn't 
> work, because I write my driver on 2.6.15, you may to DIY. User 
> response: I want a refund, because you said your hardware has Linux 
> support, but its a false statement.
> - Just leave Linux, who cares, it doens't make sense to us. Because it 
> doesn't make sense to go through all these problems to say "Linux 
> supported hardware", user will get refund the product if we say this on 
> the box on day one.
> - Maybe we have another way to do that, submit the driver to the 
> community and hope it to include it in the latest kernel source. 
> Wait.... but what about support for Redhat 9 and SuSe 8.2?

You may backport your driver to older kernel(s). Sometimes distro(s)
will backport your driver to older kernels if there is demand.

> - We are happy to maintain our own drivers, because we know better about 
> our hardware. We are paid to do so, we also have quality assurance 
> process with formal test tools and equipment. Don't think the community 
> can do a better core than us.

Maintain them "in-tree", not in your own corner.

> - Wake up! Why would the maintainers bother to maintain the drivers if 
> the driver development work is now back to the hardware vendor, like 
> drivers for other platform did? I think someone mis-understood the whole 
> idea is to "GET RID OF DRIVER MAINTENANCE", belive it or not, it belongs 
> to the vendor, not here. If the driver releases as GPL, you can still 
> make your own changes, but it doesn't have to be in main source tree.

Yeah, yeah. I just wrestled with 2 so called "GDI" printers for Windows
from 2 different vendors. Both vendors _refused to fix obvious bugs_
in their Windows drivers. Do you want THIS type of driver maintenance
to occur in Linux world too?
--
vda

Re: Linux drivers management

[Andrew James Wade]

On Tuesday 07 February 2006 14:45, David Chow wrote:

Speaking as a Linux enthusiast:

> Technical End-Users:
> - Want to compile the drivers from source
     I want to be able to compile drivers from source. I'm not particularly
interested in actually doing so as I'm not mucking around with driver
source. I happen to be compiling drivers because I am interested in
mucking around with Linux kernel source, and the drivers are in-tree. But
if the fancy takes me to play around with driver source, I want to be able
to do so. Or perhaps I want to try out a kernel in which a driver API is
being developed, in which case I need to compile drivers as source, and
having them in-tree is convenient. 

> - Enjoy building their own kernel, apply patches (patch and make, it 
> works! thats cool....)
     I enjoy building my own kernel. Applying patches, not so much. I found
applying patches to get the latest -mm drudge work and I'm never able to
remember whether 2.6.16-rc2-mm1.bz2 applies to 2.6.16-rc2 or 2.6.16.
Fortunately I found a little utility called ketchup that handles the
details for me.

> - I don't mind to search for drivers and do it myself, because it was 
> fun to make something work with my effort :) .
     And here you go off the mark. It might be fun making that device work,
but if I'm working away at a different puzzle it'll probably be just an
annoyance. When it comes to parts of a system I'm not interested in at
the moment, I want them to "just work".

> - I don't mind to upgrade my OS because of a missing driver or needed 
> for new fucntionality. Even my application breaks, down time is not 
> important to my system because it s a sytsem for fun.
     Some down time is ok for me: I don't need 100% up-time on my system.
I can accept that the cost of running a beta system (-mm kernels) is that
it occasionally crashes, and the filesystems occasionally eat data (it's
happened to me once), but it's still a nuisance. People like me are
volunteers, if it become too inconvenient we'll simply stop volunteering.
     The barrier to entry for people like us also needs to be low. And here
the situation for the kernel is fairly good (due to the stable userspace
API). When switching to a development kernel the only other thing I
had to change was lilo; everything else just worked. 

     If the Linux development community is to benefit from volunteer
testers, hardware has to work not just for the stable kernels, but also
development kernels.

     As an aside, there is another good reason to update drivers not just
for stable driver APIs, but also APIs under development: quite apart
from testing, implementing APIs is a good way to find problems in the
design of the API. Notice the reluctance of the kernel maintainers to
merge any API that is not both implemented and used.

Andrew Wade

Re: Linux drivers management

[Nicolas Mailhot]

[-- Attachment #1: Type: text/plain, Size: 2595 bytes --]

> Hi,
> On 6 February 2006 21:31, Nicolas Mailhot wrote:
> > 
> > 
> > Since you invoke end-users I'll answer.
> > 
> > This end-user is mad at hell at people like you that advocate separating
> > drivers from mainline.
> Huh...
> > Do you really think us end-users enjoy hunting your drivers all over the
> > net because you never bothered pushing them to mainline ?
> He don't needs to "bother".

Sometimes yes.
But even in these cases if *he* does not want to merge in mainline *he*
needn't invoke end-user wishes to justify himself. (I didn't answer its
other points. I *did* answer the "end-users want drivers to be separate"
bit)

> He wrote the drivers. And you never paid him.

If he was paid or helped by the hardware manufacturer to write the
driver I *did* pay him

> (Take it, "this software is beer-free" overusers, straight in your face).
> > 
> [Loads of "I'm actually proving his point, but I want to be in mainstream, so..." \
> skipped] You have just nicely proved all the stable API pros. 

Nah.
I've just proved taking drivers out-of-tree only pushes problems to
end-users. You are right when drivers are in mainline some of the burden
is reported to driver writers only:

1. they have the free support of other kernel people (reviews,
mentoring, adaptations to some kernel changes)
2. they have the free support of the kernel infrastructure (issue
system, mailing lists, hosting, testing)
3. they have the free support of distributions that package their
changes automatically
4. they can actually influence the decisions that will change the kernel
bits they depend on (just by being their their code is taken into
account)
5. they have tools knowledge and docs to deal with it
6. they are focused on their drivers while end-users have a whole system
to care about and little time to master each of its parts

And anyway do you really think a poll which asks end-users whether they
want stuff to be done by device writers or by themselves will find a
majority for "push problems to users" motions ?

The original assertion I responded to being "users clamour for
out-of-tree drivers"

You may want kernel rules change to accommodate your activity as driver
writer (just guessing, you're not answering like a user). Only :
1. do not hide between end-users you won't find them sympathetic to your
cause. Especially if you try to make them hostage to your claims.
2. the rules where written by a boatload of other kernel people so I
doubt they'd be so hard on you if you tried to follow them

-- 
Nicolas Mailhot

[-- Attachment #2: Ceci est une partie de message numériquement signée --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Linux drivers management

[Nicolas Mailhot]

[-- Attachment #1: Type: text/plain, Size: 2405 bytes --]


> I think I am in a different position like you guys, I've been work with 
> Linux from programmer level to Linux promotion . My goal is not just 
> focus on Linux technical or programming, I would like to promote this 
> operating system to not just for programmers, but also non-technical 
> end-users .

Since you invoke end-users I'll answer.

This end-user is mad at hell at people like you that advocate separating
drivers from mainline.

Do you really think us end-users enjoy hunting your drivers all over the
net because you never bothered pushing them to mainline ?

Do you really think we enjoy clicking though boatloads of HTML/js/flash
forms that will inform us about vastly important things like your custom
license, the mirror list you want us to master or your dog's birthday ?

Do you really think we enjoy learning all your out-of-tree driver
release and build processes because you couldn't be bothered with using
the same one as the kernel ?

Do you really think we enjoy locating the patch that will "fix" your
driver for our kernel because you do not bother testing anything else
than a few kernel releases, and that only for a few months after a you
wrote your driver ?

Do you really think we enjoy having out out-of-tree drivers stomp on
each-other in their eagerness to downgrade parts our working kernel to
whatever broken and obsolete version the developer happened to test ?

Do you really think we enjoy navigating support forums to find out who's
responsible for the mess ?

Do you really think we enjoy leaving in fear of a system update because
the first thing to break will be your out-of-tree drivers ?

When a driver is part of mainline it'll be in the distro kernel. It'll
be autosetup by distro tools. It'll be auto-updated by system tools. Me
the end user won't even have to know there is a driver involved -
everything will "just work".

Be honest and invoke developer laziness if you want. Invoke the utter
lack of interest of some developers in packaging or making their stuff
working on anything but their own box. Invoke their fear of a thorough
review process. Point out that they are paid to deliver a pile of code
by companies that care little if it's used or not. There are loads of
actual (bad) reasons for your demand.

But do not invoke end-users. Or end users will answer you.

-- 
Nicolas Mailhot

[-- Attachment #2: Ceci est une partie de message numériquement signée --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Linux drivers management

[Yaroslav Rastrigin]

Hi,
On 6 February 2006 21:31, Nicolas Mailhot wrote:
> 
> 
> Since you invoke end-users I'll answer.
> 
> This end-user is mad at hell at people like you that advocate separating
> drivers from mainline.
Huh...
> Do you really think us end-users enjoy hunting your drivers all over the
> net because you never bothered pushing them to mainline ?
He don't needs to "bother". He wrote the drivers. And you never paid him. 
(Take it, "this software is beer-free" overusers, straight in your face).
> 
[Loads of "I'm actually proving his point, but I want to be in mainstream, so..." skipped]
You have just nicely proved all the stable API pros. 'Cause when interfaces are clearly defined, described and implemented 
according to documentation, you won't need to even recompile (in case of ABI compatibility) or 
search for newer driver version, if your current version works nicely. You don't need to lay burden of sifting through and fixing 
metric boatloads of code when internals change onto changemaker's shoulders, 'cause everything what he needs to be concerned about is 
API compatibility. And if some particular driver worked with 2.x.xx but the same driver source fails to work on 2.x.xx+1 - 
you know where to search and what to do. And driver vendors could concentrate on improving driver cores, 
instead of rewriting interfaces. And kernel developers could concentrate on developing kernel features, instead of thinking 
"I'll introduce changes in this subsystem, this change will certainly affect IDE, SCSI, networking and may touch firewalling code" and 
spending O(N!) time modifying all affected device drivers' source by himself, instead of O(N)/O(1). And average patchset 
size will shrink by order of magnitude for medium changes. And...


> 
> But do not invoke end-users. Or end users will answer you.
> 
They ARE answering already. 
-- 
Managing your Territory since the dawn of times ...

Re: Linux drivers management

[Joshua Kugler]

On Monday 06 February 2006 09:31, Nicolas Mailhot wrote:
> > I think I am in a different position like you guys, I've been work with
> > Linux from programmer level to Linux promotion . My goal is not just
> > focus on Linux technical or programming, I would like to promote this
> > operating system to not just for programmers, but also non-technical
> > end-users .
>
> Since you invoke end-users I'll answer.

I heartily agree with this!!

I use two products that use out-of-tree drivers.  VMWare and NVidia cards.  
Fortunately, the build processes for both are rather painless, but there have 
been times when it has *not* been, and it was extremely frustrating.  I 
remember when VMWare was not doing a good job of supporting 2.6 kernels and I 
spent the better part of two days trying to track down a solution.  I finally 
did, but it was a third party, non-VMWare, patch to the VMWare code that 
fixed it so it would compile and run.  That's not what I consider convenience 
for the non-technical user.  A non-technical user would not have been able to 
do what I did, especially when they just want their software to work.

> Do you really think we enjoy clicking though boatloads of HTML/js/flash
> forms that will inform us about vastly important things like your custom
> license, the mirror list you want us to master or your dog's birthday ?

I want to install my machine and have everything work.  Don't make me chase 
all over the net trying to find a driver for my hardware.  If it's a network 
(i.e. ethernet device) the driver had *better* be in the tree.  Trying to 
download the driver to another computer, transferring, etc, is enough to make 
me find another brand of network card.

> Do you really think we enjoy learning all your out-of-tree driver
> release and build processes because you couldn't be bothered with using
> the same one as the kernel ?

Latest kernel == latest driver.  No need for me to try to keep all my drivers 
up to date.

> Do you really think we enjoy locating the patch that will "fix" your
> driver for our kernel because you do not bother testing anything else
> than a few kernel releases, and that only for a few months after a you
> wrote your driver ?

See comment about VMWare above.

> Do you really think we enjoy leaving in fear of a system update because
> the first thing to break will be your out-of-tree drivers ?

I sometimes delay kernel updates because I don't want to mess with updating my 
NVidia and VMWare drivers.  This is *not* good for security.

> But do not invoke end-users. Or end users will answer you.

So I did.  Please put your driver in the tree.  It will be better for all 
concerned.

j----- k-----

-- 
Joshua Kugler                 PGP Key: http://pgp.mit.edu/
CDE System Administrator             ID 0xDB26D7CE
http://distance.uaf.edu/

Re: Linux drivers management

[Yaroslav Rastrigin]

Hi,
> 
> I heartily agree with this!!
> 
> I use two products that use out-of-tree drivers.  VMWare and NVidia cards.  
> Fortunately, the build processes for both are rather painless, but there have 
> been times when it has *not* been, and it was extremely frustrating.  I 
> remember when VMWare was not doing a good job of supporting 2.6 kernels and I 
> spent the better part of two days trying to track down a solution.  I finally 
> did, but it was a third party, non-VMWare, patch to the VMWare code that 
> fixed it so it would compile and run.  That's not what I consider convenience 
> for the non-technical user.  A non-technical user would not have been able to 
> do what I did, especially when they just want their software to work.
And then think, why do you need to _build_ drivers in the first place. 
Wouldn't it be better to have one vmware.ko which insmod's with all 2.6 versions , from 2.6.0 to 2.6.16-rc2 , 
and throw "upgrade pain" away completely ? 
> 
> I want to install my machine and have everything work.  Don't make me chase 
> all over the net trying to find a driver for my hardware.  If it's a network 
All over the net ? Again, you're proving stable API/ABI supporters nicely. 
If kernel has stable ABI, basic/default driver is included on installation CD, and all you need to do 
is to launch ./install-linux.sh from CD in your shell or click OK and enter your root password in GUI box.
Newer/better driver - just go to device manufacturer's website, download installation package and install this driver. 
Without rebuilding. 
> (i.e. ethernet device) the driver had *better* be in the tree.  Trying to 
> download the driver to another computer, transferring, etc, is enough to make 
> me find another brand of network card.
And what to do if you've bought new hardware, installed it and _voila_ - NO IN-TREE DRIVER exists ?
Do you want every Linux user  going for shopping to nearest WalMart carry full linux hardware compatibility list printed out ?
Or intree driver list ?
> Latest kernel == latest driver.  No need for me to try to keep all my drivers 
> up to date.
Wrong. Latest kernel is latest kernel. Latest driver is latest driver. They are different entities, and don't mix'em.
> 
> I sometimes delay kernel updates because I don't want to mess with updating my 
> NVidia and VMWare drivers.  This is *not* good for security.
So who to blame ? Maybe, just look at those who don't want stable driver API ?
> So I did.  Please put your driver in the tree.  It will be better for all 
> concerned.
Please, don't force your preferences over others'
-- 
Managing your Territory since the dawn of times ...

Re: Linux drivers management

[Martin Mares]

Hello!

> And then think, why do you need to _build_ drivers in the first place.
> Wouldn't it be better to have one vmware.ko which insmod's with all
> 2.6 versions , from 2.6.0 to 2.6.16-rc2 , and throw "upgrade pain"
> away completely ? 

Well, you want the vmware.ko to contain machine code for at least 17
different architecture the kernel runs on? ;-)

> If kernel has stable ABI, basic/default driver is included on
> installation CD, and all you need to do 
[...]

... and I'm screwed if I happen to use a 2 years old network card with
written for a 2.4.x kernel, while I am running on 2.6.x.

> is to launch ./install-linux.sh from CD in your shell or click OK and
> enter your root password in GUI box. Newer/better driver - just go to
> device manufacturer's website, download installation package and
> install this driver. Without rebuilding. 

While now, I don't need to install or rebuild anything, just use the
stock kernel which already contains the drivers :)

> And what to do if you've bought new hardware, installed it and _voila_
> - NO IN-TREE DRIVER exists ?

The same like what I do if I buy new hardware and no Linux driver exist
-- call myself an uninformed ignoramus, because that's who I am.

> Please, don't force your preferences over others'

Doesn't this sentence apply to you as well? ;-)

				Have a nice fortnight
-- 
Martin `MJ' Mares   <mj@ucw.cz>   http://atrey.karlin.mff.cuni.cz/~mj/
Faculty of Math and Physics, Charles University, Prague, Czech Rep., Earth
It said, "Insert disk #3," but only two will fit!

Re: Linux drivers management

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1084 bytes --]

On Mon, 2006-02-06 20:39:37 +0100, Martin Mares <mj@ucw.cz> wrote:
> > And then think, why do you need to _build_ drivers in the first place.
> > Wouldn't it be better to have one vmware.ko which insmod's with all
> > 2.6 versions , from 2.6.0 to 2.6.16-rc2 , and throw "upgrade pain"
> > away completely ? 
> 
> Well, you want the vmware.ko to contain machine code for at least 17
> different architecture the kernel runs on? ;-)

I think we will first need to hack binutils. The last time I heared
about it, Fat Binaries weren't implemented (only containing object
code for two different architectures), let alone for _all_
architectures...

Even getting all GCCs to compile with the same version would be an
adventure...

MfG, JBG

-- 
Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481             _ O _
"Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg  _ _ O
 für einen Freien Staat voll Freier Bürger"  | im Internet! |   im Irak!   O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Linux drivers management

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 4825 bytes --]

On Mon, 2006-02-06 22:17:19 +0300, Yaroslav Rastrigin <yarick@it-territory.ru> wrote:
> > I use two products that use out-of-tree drivers.  VMWare and NVidia cards.  
> > Fortunately, the build processes for both are rather painless, but there have 
> > been times when it has *not* been, and it was extremely frustrating.  I 
> > remember when VMWare was not doing a good job of supporting 2.6 kernels and I 
> > spent the better part of two days trying to track down a solution.  I finally 
> > did, but it was a third party, non-VMWare, patch to the VMWare code that 
> > fixed it so it would compile and run.  That's not what I consider convenience 
> > for the non-technical user.  A non-technical user would not have been able to 
> > do what I did, especially when they just want their software to work.
> And then think, why do you need to _build_ drivers in the first place. 
> Wouldn't it be better to have one vmware.ko which insmod's with all 2.6 versions , from 2.6.0 to 2.6.16-rc2 , 
> and throw "upgrade pain" away completely ? 

This would only work if we sacrified the freedom to change something.
The kernel code base changes. A lot, actually. If it didn't, there eg.
wouldn't be suspend2whatever, because the API was plain missing back
in those days. So sacrifice evolution for backwards compatibility?

These days (and it has always been that way) kernel development is a
quite active process. If core-APIs need to be changed, the person who
does it usually also updates all users of the given API. That won't
work if the drivers are not in the codebase, no chance to grep for
something on 3rd vendor's websites...

> > I want to install my machine and have everything work.  Don't make me chase 
> > all over the net trying to find a driver for my hardware.  If it's a network 
> All over the net ? Again, you're proving stable API/ABI supporters nicely. 
> If kernel has stable ABI, basic/default driver is included on installation CD, and all you need to do 
> is to launch ./install-linux.sh from CD in your shell or click OK and enter your root password in GUI box.
> Newer/better driver - just go to device manufacturer's website, download installation package and install this driver. 
> Without rebuilding. 

Not everybody is using RedHar, SuSE, Debian or whatever. Consider I
was building a custom QBus-to-PCI bridge to use some ATI/NVidia
graphics board in my 15y old VAX. If my hardware hack required
broadening some in-kernel API, do you really think some guy at NVidia
(only to name an example:-) would cross-compile their stuff for a VAX?
Given a userbase of exactly _one_ person?

> > (i.e. ethernet device) the driver had *better* be in the tree.  Trying to 
> > download the driver to another computer, transferring, etc, is enough to make 
> > me find another brand of network card.
> And what to do if you've bought new hardware, installed it and _voila_ - NO IN-TREE DRIVER exists ?
> Do you want every Linux user  going for shopping to nearest WalMart carry full linux hardware compatibility list printed out ?
> Or intree driver list ?

Usually, it's quite simple to buy correct hardware. Look for something
that's a tad more intelligent (SCSI scanners in favour of USB/parport,
postscript printers, ...) and offloads the host CPU.

> > I sometimes delay kernel updates because I don't want to mess with updating my 
> > NVidia and VMWare drivers.  This is *not* good for security.
> So who to blame ? Maybe, just look at those who don't want stable driver API ?

The Linux kernel is a project (or hundreds actually) that have choosen
their way of operation. That's evolution with not a lot of
looking-back. If you want to have a stable API, heck, just prepare
another fork and implement it. If this is what users want, they'll
take it.

> > So I did.  Please put your driver in the tree.  It will be better for all 
> > concerned.
> Please, don't force your preferences over others'

My (personal!) view is that Linux isn't actually about the users. It's
about the developers. Developers develop what they have a use for (or
become famous.) Sometimes, regular users can make good use of it, once
distributions prepared all the userland.

So if you're a developer, try to become famous for implementing a
stable API.

If you're a user, stop fighting against an operating system's kernel
and start looking for a system _you_ want to use. Maybe some WinXP
variant?

MfG, JBG

-- 
Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481             _ O _
"Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg  _ _ O
 für einen Freien Staat voll Freier Bürger"  | im Internet! |   im Irak!   O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Linux drivers management

[Jesper Juhl]

On 2/6/06, Yaroslav Rastrigin <yarick@it-territory.ru> wrote:
> Hi,
> >
> > I heartily agree with this!!
> >
> > I use two products that use out-of-tree drivers.  VMWare and NVidia cards.
> > Fortunately, the build processes for both are rather painless, but there have
> > been times when it has *not* been, and it was extremely frustrating.  I
> > remember when VMWare was not doing a good job of supporting 2.6 kernels and I
> > spent the better part of two days trying to track down a solution.  I finally
> > did, but it was a third party, non-VMWare, patch to the VMWare code that
> > fixed it so it would compile and run.  That's not what I consider convenience
> > for the non-technical user.  A non-technical user would not have been able to
> > do what I did, especially when they just want their software to work.
> And then think, why do you need to _build_ drivers in the first place.
> Wouldn't it be better to have one vmware.ko which insmod's with all 2.6 versions , from 2.6.0 to 2.6.16-rc2 ,
> and throw "upgrade pain" away completely ?
> >
> > I want to install my machine and have everything work.  Don't make me chase
> > all over the net trying to find a driver for my hardware.  If it's a network
> All over the net ? Again, you're proving stable API/ABI supporters nicely.
> If kernel has stable ABI, basic/default driver is included on installation CD, and all you need to do
> is to launch ./install-linux.sh from CD in your shell or click OK and enter your root password in GUI box.

Why should I have to bother with that?
It's a lot more convenient that my new kernel just "magically"
includes the driver for my hardware, and the latest version of that
driver even. No inserting cd, run install script, navigate through
installer etc crap.


> Newer/better driver - just go to device manufacturer's website, download installation package and install this driver.

1. You are assuming all users have internet connections and can do
that - not true.
2. Again, why should I have to go through the trouble of first finding
the manufacturers website, then find out where they keep the drivers,
then download and figure out how to install the driver?
3. What if the manufacturer goes out of business and the website disappears?
4. What happens when the manufacturer stops updating the driver? Even
with an almost 100% frozen driver API changes *will* happen over time
and the driver will eventually stop working - or, if the driver API is
kept *100%* frozen then the core kernel will steadily aquire a huge
amount of backwards compatibility cruft which will eventually become a
maintainance nightmare.


> Without rebuilding.

Users of distro kernels already have the in-kernel drivers prebuild for them.
Users of custom kernels have to build the core kernel anyway, buildign
a driver or two at the same time is no big deal.

> > (i.e. ethernet device) the driver had *better* be in the tree.  Trying to
> > download the driver to another computer, transferring, etc, is enough to make
> > me find another brand of network card.
> And what to do if you've bought new hardware, installed it and _voila_ - NO IN-TREE DRIVER exists ?

Then you bought the wrong hardware.


> Do you want every Linux user  going for shopping to nearest WalMart carry full linux hardware compatibility list printed out ?
> Or intree driver list ?

Users who prefer not to have hardware trouble would be wise to check
compatibility before purchasing. This goes not only for Linux, but for
any OS - and don't say "all hardware has manufacturer supplied drivers
for Windows" 'cos that's not true, and there are several incompatible
windows versions, just try loading WinXP drivers on Windows98 or a
WinNT driver on Windows Server 2003 or any other fun combination.


> > Latest kernel == latest driver.  No need for me to try to keep all my drivers
> > up to date.
> Wrong. Latest kernel is latest kernel. Latest driver is latest driver. They are different entities, and don't mix'em.
> >
When the driver is maintained in-tree, latest kernel == latest driver.


> > I sometimes delay kernel updates because I don't want to mess with updating my
> > NVidia and VMWare drivers.  This is *not* good for security.
> So who to blame ?

NVidia and VMWare obviously for not submitting Open Source drivers for
inclusion in mainline.


> Maybe, just look at those who don't want stable driver API ?
> > So I did.  Please put your driver in the tree.  It will be better for all
> > concerned.
> Please, don't force your preferences over others'

Nobody is forcing anybody to do anything. We are just not bending over
backwards for people who refuse to work with us in-tree. If someone
wants to maintain drivers out-of-tree they are perfectly free to do
so, they just don't get the bennefit of having their drivers
auto-updated when API's change and they don't get the bennefit of many
develoers improving their drivers, they have to do *all* the work
themselves - but that's their choice, noone forced it on them.


--
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: Linux drivers management

[Bernd Petrovitsch]

On Mon, 2006-02-06 at 22:17 +0300, Yaroslav Rastrigin wrote:
[...]
> All over the net ? Again, you're proving stable API/ABI supporters nicely. 

Not necessarily since there are other solutions like "submit the driver
into the kernel".
And exactly then you get the best of both worlds: The driver is "up to
date" and not even distributors have care (well, at least for almost all
of them).

> If kernel has stable ABI, basic/default driver is included on installation CD, and all you need to do 

Have fun done *doing* this.

> And what to do if you've bought new hardware, installed it and _voila_ - NO IN-TREE DRIVER exists ?
> Do you want every Linux user  going for shopping to nearest WalMart carry full linux hardware compatibility list printed out ?

It worked years ago that way. So what?

	Bernd
-- 
Firmix Software GmbH                   http://www.firmix.at/
mobil: +43 664 4416156                 fax: +43 1 7890849-55
          Embedded Linux Development and Services

Re: Linux drivers management

[linux-os (Dick Johnson)]

On Mon, 6 Feb 2006, Joshua Kugler wrote:

> On Monday 06 February 2006 09:31, Nicolas Mailhot wrote:
>>> I think I am in a different position like you guys, I've been work with
>>> Linux from programmer level to Linux promotion . My goal is not just
>>> focus on Linux technical or programming, I would like to promote this
>>> operating system to not just for programmers, but also non-technical
>>> end-users .
>>
>> Since you invoke end-users I'll answer.
>
> I heartily agree with this!!
>
> I use two products that use out-of-tree drivers.  VMWare and NVidia cards.
> Fortunately, the build processes for both are rather painless, but there have
> been times when it has *not* been, and it was extremely frustrating.  I
> remember when VMWare was not doing a good job of supporting 2.6 kernels and I
> spent the better part of two days trying to track down a solution.  I finally
> did, but it was a third party, non-VMWare, patch to the VMWare code that
> fixed it so it would compile and run.  That's not what I consider convenience
> for the non-technical user.  A non-technical user would not have been able to
> do what I did, especially when they just want their software to work.
>
>> Do you really think we enjoy clicking though boatloads of HTML/js/flash
>> forms that will inform us about vastly important things like your custom
>> license, the mirror list you want us to master or your dog's birthday ?
>
> I want to install my machine and have everything work.  Don't make me chase
> all over the net trying to find a driver for my hardware.  If it's a network
> (i.e. ethernet device) the driver had *better* be in the tree.  Trying to
> download the driver to another computer, transferring, etc, is enough to make
> me find another brand of network card.
>
>> Do you really think we enjoy learning all your out-of-tree driver
>> release and build processes because you couldn't be bothered with using
>> the same one as the kernel ?
>
> Latest kernel == latest driver.  No need for me to try to keep all my drivers
> up to date.
>
>> Do you really think we enjoy locating the patch that will "fix" your
>> driver for our kernel because you do not bother testing anything else
>> than a few kernel releases, and that only for a few months after a you
>> wrote your driver ?
>
> See comment about VMWare above.
>
>> Do you really think we enjoy leaving in fear of a system update because
>> the first thing to break will be your out-of-tree drivers ?
>
> I sometimes delay kernel updates because I don't want to mess with updating my
> NVidia and VMWare drivers.  This is *not* good for security.
>
>> But do not invoke end-users. Or end users will answer you.
>
> So I did.  Please put your driver in the tree.  It will be better for all
> concerned.
>
> j----- k-----
>
> --
> Joshua Kugler                 PGP Key: http://pgp.mit.edu/
> CDE System Administrator             ID 0xDB26D7CE
> http://distance.uaf.edu/


Maybe it would be better for no drivers to be in the tree!
Something along the lines of an automatic FTP site that
interacts with a configuration program. You end up downloading
the drivers that you need. In the case where you don't have
an Internet connection, a distribution company would put the
mirror on a CD or DVD.

Right now, there are really too many drivers in the kernel.
The kernel should have a stable API for drivers and they
should be in a separate tree, either on the Web or on a
distribution disc. There are many drivers that are as old
as Linux! The 3c501.c and 3c503.c are examples. You can't
remove them from the kernel without invoking a thousand
angry responses. These boards and the ne*.c network boards
just won't go away!

This means that the amount of drivers will continue to
increase to, eventually, an unmanagable amount. This is
why they really need to be seperately managed!


Cheers,
Dick Johnson
Penguin : Linux version 2.6.13.4 on an i686 machine (5589.66 BogoMips).
Warning : 98.36% of all statistics are fiction.
_


****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Re: Linux drivers management

[Michael Krufky]

linux-os (Dick Johnson) wrote:

>Maybe it would be better for no drivers to be in the tree!
>Something along the lines of an automatic FTP site that
>interacts with a configuration program. You end up downloading
>the drivers that you need. In the case where you don't have
>an Internet connection, a distribution company would put the
>mirror on a CD or DVD.
>  
>
Regardless of whether or not the idea is practical, where would the 
funding come from?  Who is going to donate their time?  What if they get 
bored of it and nobody wants to pick up?

There are enough of us working for free on this stuff, and those that 
get paid already have enough on their plate.  What you're asking for is 
something that just isn't practical.

When I first read this, I thought you were joking... unfortunately, it 
looks like you are being serious.

>Right now, there are really too many drivers in the kernel.
>The kernel should have a stable API for drivers and they
>should be in a separate tree, either on the Web or on a
>distribution disc. There are many drivers that are as old
>as Linux! The 3c501.c and 3c503.c are examples. You can't
>remove them from the kernel without invoking a thousand
>angry responses. These boards and the ne*.c network boards
>just won't go away!
>  
>
Why would we WANT to remove them?  Linux is just about the only 
operating system that will run well on all old machines.  If we were to 
remove these drivers from the kernel, we might as well all throw our old 
hardware into the garbage.

>This means that the amount of drivers will continue to
>increase to, eventually, an unmanagable amount. This is
>why they really need to be seperately managed!
>  
>
That's part of what subsystems and subsystem maintainers are for.  Looks 
like somebody thought ahead. ;-)

FYI, v4l/dvb drivers can be built separately from the kernel as modules, 
directly from our mercurial repository, and we try to keep them all 
backwards compatable with older vanilla kernels.  Surely there are other 
subsystems that do something similar.

>****************************************************************
>The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.
>
^^^ I'm sure you've been flamed about this already, so I won't say it....

Cheers,

Michael Krufky

Re: Linux drivers management

[Nicolas Mailhot]

[-- Attachment #1: Type: text/plain, Size: 737 bytes --]

Le lundi 06 février 2006 à 14:21 -0500, linux-os (Dick Johnson) a
écrit :

> Right now, there are really too many drivers in the kernel.
> The kernel should have a stable API for drivers and they
> should be in a separate tree, either on the Web or on a
> distribution disc. There are many drivers that are as old
> as Linux! The 3c501.c and 3c503.c are examples. You can't
> remove them from the kernel without invoking a thousand
> angry responses. These boards and the ne*.c network boards
> just won't go away!

This is another proof in-kernel maintenance is cheaper.
If out-of-tree drivers where lower maintenance like it's claimed, they'd
have a lower attrition rate than in-kernel stuff.

-- 
Nicolas Mailhot

[-- Attachment #2: Ceci est une partie de message numériquement signée --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Linux drivers management

[Gene Heskett]

On Monday 06 February 2006 13:31, Nicolas Mailhot wrote:
>> I think I am in a different position like you guys, I've been work
>> with Linux from programmer level to Linux promotion . My goal is not
>> just focus on Linux technical or programming, I would like to
>> promote this operating system to not just for programmers, but also
>> non-technical end-users .
>
>Since you invoke end-users I'll answer.

So will I, but this message says it better than one of my rants would.
So I'll just second it and add:

Get used to it David, you are NOT going to change it with anything 
resembling your current arguments, which IMO do nothing but waste 
bandwidth, and busy peoples time.  Go away, way away.  Go sabotage a 
winderz project and leave us the hell alone.

>This end-user is mad at hell at people like you that advocate
> separating drivers from mainline.
>
>Do you really think us end-users enjoy hunting your drivers all over
> the net because you never bothered pushing them to mainline ?
>
>Do you really think we enjoy clicking though boatloads of
> HTML/js/flash forms that will inform us about vastly important things
> like your custom license, the mirror list you want us to master or
> your dog's birthday ?
>
>Do you really think we enjoy learning all your out-of-tree driver
>release and build processes because you couldn't be bothered with
> using the same one as the kernel ?
>
>Do you really think we enjoy locating the patch that will "fix" your
>driver for our kernel because you do not bother testing anything else
>than a few kernel releases, and that only for a few months after a you
>wrote your driver ?
>
>Do you really think we enjoy having out out-of-tree drivers stomp on
>each-other in their eagerness to downgrade parts our working kernel to
>whatever broken and obsolete version the developer happened to test ?
>
>Do you really think we enjoy navigating support forums to find out
> who's responsible for the mess ?
>
>Do you really think we enjoy leaving in fear of a system update
> because the first thing to break will be your out-of-tree drivers ?
>
>When a driver is part of mainline it'll be in the distro kernel. It'll
>be autosetup by distro tools. It'll be auto-updated by system tools.
> Me the end user won't even have to know there is a driver involved -
> everything will "just work".
>
>Be honest and invoke developer laziness if you want. Invoke the utter
>lack of interest of some developers in packaging or making their stuff
>working on anything but their own box. Invoke their fear of a thorough
>review process. Point out that they are paid to deliver a pile of code
>by companies that care little if it's used or not. There are loads of
>actual (bad) reasons for your demand.
>
>But do not invoke end-users. Or end users will answer you.

-- 
Cheers, Gene
People having trouble with vz bouncing email to me should add the word
'online' between the 'verizon', and the dot which bypasses vz's
stupid bounce rules.  I do use spamassassin too. :-)
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2006 by Maurice Eugene Heskett, all rights reserved.

Linux drivers management

[David Chow]

Dear maintainers,

Is there any work in Linux undergoing to separate Linux drivers and the 
the main kernel, and manage drivers using a package management system 
that only manages kernel drivers and modules? If this can be done, the 
kernel maintenance can be simple, and will end-up with a more stable 
(less frequent changed) kernel API for drivers, also make every 
developers of drivers happy.

Would like to see that happens .

regards,
David Chow

Re: Linux drivers management

[Michal Schmidt]

David Chow wrote:
> Dear maintainers,
> 
> Is there any work in Linux undergoing to separate Linux drivers and the 
> the main kernel, and manage drivers using a package management system 
> that only manages kernel drivers and modules? If this can be done, the 
> kernel maintenance can be simple, and will end-up with a more stable 
> (less frequent changed) kernel API for drivers, also make every 
> developers of drivers happy.
> 
> Would like to see that happens .

Please read Documentation/stable_api_nonsense.txt in your copy of Linux 
source.

Michal

Re: Linux drivers management

[David Chow]

> Please read Documentation/stable_api_nonsense.txt in your copy of 
> Linux source.
I've read the document, I strongly disagree, because it is not relavant 
to my question or to my original purpose of this question.

Putting the driver source code in the kernel source tree has nothing to 
do with talking about a stable kernel API . Even you put the driver 
sources into the main kernel tree, it will still need a lot of work to 
port all drivers if the API changes. Driver sources can still host in a 
different project (e.g. projects in sf.net) and maintain open-source and 
om by the community, no difference than before

For different compile time options that affect data structures, this is 
well known a bad idea . These types of techniques no longer allowed in 
Java and other OO languages . Because I can simply say the code is not 
portable. If really need a recompile and optimize, the distro vendor 
should bare this, but according to the document, "As Linux supports a 
larger number of different devices "out of the box" than any other 
operating system" , do you think Linux should one day or some day grow 
to 1TB source tree to include all possible drivers for all hw come from 
the world? I don't see there is reason why a kernel or OS need to 
include all the drivers for all the hardware. I don't think there is any 
OS vendors on the market to capable to distribute all drivers integrity, 
then the choice is to make a disabled Linux OS because of an OSV who has 
only limited supporting resources to suppport and certify limited 
hardware devices.

Please see my other email responded to Jes about the learning curve and 
documentation issues of a Linux driver developer to pick up Linux skills.

regards,
David Chow

Re: Linux drivers management

[Randy.Dunlap]

On Tue, 7 Feb 2006, David Chow wrote:

>
> > Please read Documentation/stable_api_nonsense.txt in your copy of
> > Linux source.
> I've read the document, I strongly disagree, because it is not relavant
> to my question or to my original purpose of this question.
>
> Putting the driver source code in the kernel source tree has nothing to
> do with talking about a stable kernel API . Even you put the driver
> sources into the main kernel tree, it will still need a lot of work to
> port all drivers if the API changes. Driver sources can still host in a
> different project (e.g. projects in sf.net) and maintain open-source and
> om by the community, no difference than before
>
> For different compile time options that affect data structures, this is
> well known a bad idea . These types of techniques no longer allowed in
> Java and other OO languages . Because I can simply say the code is not
> portable. If really need a recompile and optimize, the distro vendor
> should bare this, but according to the document, "As Linux supports a
> larger number of different devices "out of the box" than any other
> operating system" , do you think Linux should one day or some day grow
> to 1TB source tree to include all possible drivers for all hw come from
> the world? I don't see there is reason why a kernel or OS need to
> include all the drivers for all the hardware. I don't think there is any
> OS vendors on the market to capable to distribute all drivers integrity,
> then the choice is to make a disabled Linux OS because of an OSV who has
> only limited supporting resources to suppport and certify limited
> hardware devices.
>
> Please see my other email responded to Jes about the learning curve and
> documentation issues of a Linux driver developer to pick up Linux skills.

Maybe you want something like DKMS from Dell?
  http://linux.dell.com/projects.shtml

or maybe some of the distros have something that fits your needs.

-- 
~Randy

Re: Linux drivers management

[Alan Cox]

On Maw, 2006-02-07 at 00:50 +0800, David Chow wrote:
> do with talking about a stable kernel API . Even you put the driver 
> sources into the main kernel tree, it will still need a lot of work to 
> port all drivers if the API changes. 

Convention is that he who breaks an API fixes up the majority of the
mess caused or does it by consensus with the other developers. 

Of course if your code is out of tree nobody will know so it'll just
break.

For example the last time I edited the wdt501 watchdog I submitted
according to my logs is about 1998. Since then each person who broke an
API it used fixed it up, or the janitors did shortly afterwards. 

> For different compile time options that affect data structures, this is 
> well known a bad idea .

Do you have measured performance data, economic models and statistical
evidence to back up the claim or is this that well known bad idea called
"conventional wisdom" ?

> operating system" , do you think Linux should one day or some day grow 
> to 1TB source tree to include all possible drivers for all hw come from 
> the world? 

Why not. By the time it gets that big you'll have over 1TB on your phone
let alone a PC. The kernel development structure is optimised for
development. 

Alan

Re: Linux drivers management

[Jesper Juhl]

On 2/6/06, David Chow <davidchow@shaolinmicro.com> wrote:
>
> > Please read Documentation/stable_api_nonsense.txt in your copy of
> > Linux source.
> I've read the document, I strongly disagree, because it is not relavant
> to my question or to my original purpose of this question.
>
It has a lot of relevance.
If you split up drivers and the core kernel into two sepperate
projects I can easily imagine the two getting out of sync whenever an
API change needs to be made.
Currently whenever someone changes a kernel API that person is
required to also update all users of that API. Since all users are
in-tree that's resonably easy to do. If the drivers are out-of-tree
it's a *lot* harder for several reasons;

a) to fix users of the API I can no longer just 'cd drivers/' I now
have to go download & extract a sepperate source tree.

b) if an API change I make gets accepted by the 'core kernel' team but
the 'driver-tree' maintainers refuse my updates we get out-of-sync -
just as if the 'core kernel' tree rejects the API change but the
'driver-tree' maintainers already merged the update to the drivers to
use the new API.

c) presumably the two projects 'driver-tree' and 'core-kernel' would
use sepperate mailing lists, making it harder to communicate about API
changes.

Maintaining the drivers out of tree would make API changes a lot more
painful and thus we'd need to lock down the API a lot harder than we
do today. This in turn would mean that some types of bugs will be
harder to solve (since a proper fix would involve a painful API
change) and advancement of kernel caabilities/features etc will be
slower.


> Putting the driver source code in the kernel source tree has nothing to
> do with talking about a stable kernel API . Even you put the driver
> sources into the main kernel tree, it will still need a lot of work to
> port all drivers if the API changes.

You can't dodge the work that needs to be done when API's change, but
you /can/ try to make the work as easy to do as possible by keeping
the core kernel code and the drivers close together and maintained as
a single unit.


> Driver sources can still host in a
> different project (e.g. projects in sf.net) and maintain open-source and
> om by the community, no difference than before
>
> For different compile time options that affect data structures, this is
> well known a bad idea . These types of techniques no longer allowed in
> Java and other OO languages .

The kernel is written in C. What other languages do/allow/recommend
etc is irrelevant.


>Because I can simply say the code is not
> portable. If really need a recompile and optimize, the distro vendor
> should bare this, but according to the document, "As Linux supports a
> larger number of different devices "out of the box" than any other
> operating system" , do you think Linux should one day or some day grow
> to 1TB source tree to include all possible drivers for all hw come from
> the world?

First of all, drivers are regularly dropped from the tree; either
because they become unmaintained and bitrot or because the hardware
becomes extremely obsolete and rare.
Secondly, I predict that available storage sizes and bandwith
available to users of the kernel will grow faster than the size of the
source tree (and cost of storage & bandwith will likely continue to
drop as well).
Thirdly, the day Linux supports "all hw come from the world" I'll be
dancing with joy.


> I don't see there is reason why a kernel or OS need to
> include all the drivers for all the hardware.

This is *extremely* convenient for our users.


> I don't think there is any
> OS vendors on the market to capable to distribute all drivers integrity,
> then the choice is to make a disabled Linux OS because of an OSV who has
> only limited supporting resources to suppport and certify limited
> hardware devices.
>
The user always has the option of building any driver they need
themselves. This is easy since the drivers are all in the main kernel
source tree, the user doesn't have to go hunt for them online
(assuming the user even has an internet connection).


> Please see my other email responded to Jes about the learning curve and
> documentation issues of a Linux driver developer to pick up Linux skills.
>
The fact that all documentation (well, at least a lot of it) is kept
in a central place (Documentation/) is a very nice thing for someone
trying to learn their way around the kernel - I know I've personally
bennefitted from that.
Also the fact that core kernel code, drivers and supporting scripts
are all kept in a single source tree is very convenient for new
developers since there's only one thing to download and then you have
a complete tree with the full picture to learn from - again I say this
based on personal experience.


But in any case, if you want to maintain one or more drivers
out-of-tree, then go ahead, noone's stopping you. But your maintenance
work will be a lot lower if you instead submit your drivers for
inclusion in mainline since then other people will keep your driver
up-to-date and in sync with API changes when they happen. You might
even get a few bugs fixed for free and get the code cleaned up for
free.


--
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: Linux drivers management

[Jes Sorensen]

>>>>> "David" == David Chow <davidchow@shaolinmicro.com> writes:

David> Dear maintainers, Is there any work in Linux undergoing to
David> separate Linux drivers and the the main kernel, and manage
David> drivers using a package management system that only manages
David> kernel drivers and modules? If this can be done, the kernel
David> maintenance can be simple, and will end-up with a more stable
David> (less frequent changed) kernel API for drivers, also make every
David> developers of drivers happy.

David> Would like to see that happens .

Simple answer: no

Maybe someone is working on it, but it's highly unlikely to be
anything but a waste of that person's time.

This is a classic question, by seperating out the drivers you make it
so much harder for all developers to propagate changes into all pieces
of the tree.

Cheers,
Jes

Re: Linux drivers management

[David Chow]

>David> separate Linux drivers and the the main kernel, and manage
>David> drivers using a package management system that only manages
>David> kernel drivers and modules? If this can be done, the kernel
>David> maintenance can be simple, and will end-up with a more stable
>David> (less frequent changed) kernel API for drivers, also make every
>David> developers of drivers happy.
>
>David> Would like to see that happens .
>
>Simple answer: no
>
>Maybe someone is working on it, but it's highly unlikely to be
>anything but a waste of that person's time.
>
>This is a classic question, by seperating out the drivers you make it
>so much harder for all developers to propagate changes into all pieces
>of the tree.
I write drivers, never need to change kernel if the kernel API is mature 
enough to provide the need of a module developer needs. There is no 
reason to make changes to the kernel source, only needed because the 
original kernel code is crap or the API designed without proper 
software/system architectural design work effort. Each Linux kernel 
version go through a lengthy beta release cycle (e.g. 2.3, 2.5, 2.7), 
this shouldn't happen and idea collection should be enough through this 
large Linux community.

If our time is to focus on kernel's kernel, writing good documentation 
about a stable kernel API, it will benefit many developers to write 
drivers to Linux . It is too difficult to learn, this is a main reason 
why Linux is lack of support from manufacturer drivers, not because they 
don't like Linux and no market, it is because this has created high 
entry barrier for them.

I've been working on Linux modules for many years, training my 
engineers, talking to developers, hw manufacturers .. believe it or not, 
this is the main reason. They all ask for a DDK for Linux that can make 
drivers easily for their product.

I think I am in a different position like you guys, I've been work with 
Linux from programmer level to Linux promotion . My goal is not just 
focus on Linux technical or programming, I would like to promote this 
operating system to not just for programmers, but also non-technical 
end-users . Writing C code to me is just bits of task of some process.  
You are too much focus on programming without considering the market 
situation.

There is no right or wrong for this question, but my original question 
is to listen thoughts and to hear the goal of people in the list. And of 
course, I would really like to see you people look into the way to 
facilitate more people gets a path with ease to Linux drivers 
development. User driver installation without the need to know about 
kernel sources, gcc, make etc....  "Because I am a dummy, I want to 
plug-in my device, put in the driver disc and hope it works!"

regards,
David Chow

Re: Linux drivers management

[Pedro Alves]

[-- Attachment #1: Type: text/plain, Size: 3526 bytes --]

David,

   You got my vote..... I make simple hardware to improve availability 
of operational systems and i just
can't go to build a native driver because i could not find a kit that 
works.... now i use a perl script to
acomplish this mission... it works greate but is not beaulty as i like 
it to be.... Time is money.....

  Pedro Alves

David Chow wrote:

>
>> David> separate Linux drivers and the the main kernel, and manage
>> David> drivers using a package management system that only manages
>> David> kernel drivers and modules? If this can be done, the kernel
>> David> maintenance can be simple, and will end-up with a more stable
>> David> (less frequent changed) kernel API for drivers, also make every
>> David> developers of drivers happy.
>>
>> David> Would like to see that happens .
>>
>> Simple answer: no
>>
>> Maybe someone is working on it, but it's highly unlikely to be
>> anything but a waste of that person's time.
>>
>> This is a classic question, by seperating out the drivers you make it
>> so much harder for all developers to propagate changes into all pieces
>> of the tree.
>
> I write drivers, never need to change kernel if the kernel API is 
> mature enough to provide the need of a module developer needs. There 
> is no reason to make changes to the kernel source, only needed because 
> the original kernel code is crap or the API designed without proper 
> software/system architectural design work effort. Each Linux kernel 
> version go through a lengthy beta release cycle (e.g. 2.3, 2.5, 2.7), 
> this shouldn't happen and idea collection should be enough through 
> this large Linux community.
>
> If our time is to focus on kernel's kernel, writing good documentation 
> about a stable kernel API, it will benefit many developers to write 
> drivers to Linux . It is too difficult to learn, this is a main reason 
> why Linux is lack of support from manufacturer drivers, not because 
> they don't like Linux and no market, it is because this has created 
> high entry barrier for them.
>
> I've been working on Linux modules for many years, training my 
> engineers, talking to developers, hw manufacturers .. believe it or 
> not, this is the main reason. They all ask for a DDK for Linux that 
> can make drivers easily for their product.
>
> I think I am in a different position like you guys, I've been work 
> with Linux from programmer level to Linux promotion . My goal is not 
> just focus on Linux technical or programming, I would like to promote 
> this operating system to not just for programmers, but also 
> non-technical end-users . Writing C code to me is just bits of task of 
> some process.  You are too much focus on programming without 
> considering the market situation.
>
> There is no right or wrong for this question, but my original question 
> is to listen thoughts and to hear the goal of people in the list. And 
> of course, I would really like to see you people look into the way to 
> facilitate more people gets a path with ease to Linux drivers 
> development. User driver installation without the need to know about 
> kernel sources, gcc, make etc....  "Because I am a dummy, I want to 
> plug-in my device, put in the driver disc and hope it works!"
>
> regards,
> David Chow
> -
> To unsubscribe from this list: send the line "unsubscribe 
> linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>
>


[-- Attachment #2: pedro.vcf --]
[-- Type: text/x-vcard, Size: 380 bytes --]

begin:vcard
fn:Pedro Alves
n:Alves;Pedro
org:Moebius Tecnologia em Informatica Ltda
adr:;;Rua Jardim Botanico 674 sala 507;Rio de janeiro;RJ;22461000;Brasil
email;internet:pedro@moebius.com.br
title:Diretor de Marketing
tel;work:21 2294-3772
tel;fax:21 2294-2794
tel;home:21 8762-1264
tel;cell:21 8762-1264
x-mozilla-html:TRUE
url:http://www.moebius.com.br
version:2.1
end:vcard

Re: Linux drivers management

[Geert Uytterhoeven]

On Tue, 7 Feb 2006, David Chow wrote:
> There is no right or wrong for this question, but my original question is to
> listen thoughts and to hear the goal of people in the list. And of course, I
> would really like to see you people look into the way to facilitate more
> people gets a path with ease to Linux drivers development. User driver
> installation without the need to know about kernel sources, gcc, make etc....
> "Because I am a dummy, I want to plug-in my device, put in the driver disc and
> hope it works!"

Because I'm not a dummy, I want to plug-in my device, and it will work.

Gr{oetje,eeting}s,

						Geert

P.S. Which driver disc? On which format? Anyone still has a floppy drive?
     Where's the driver to my memory card reader?
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: Linux drivers management

[Jes Sorensen]

David Chow wrote:
>> This is a classic question, by seperating out the drivers you make it
>> so much harder for all developers to propagate changes into all pieces
>> of the tree.
> 
> I write drivers, never need to change kernel if the kernel API is mature 
> enough to provide the need of a module developer needs. There is no 
> reason to make changes to the kernel source, only needed because the 
> original kernel code is crap or the API designed without proper 
> software/system architectural design work effort. Each Linux kernel 
> version go through a lengthy beta release cycle (e.g. 2.3, 2.5, 2.7), 
> this shouldn't happen and idea collection should be enough through this 
> large Linux community.

Silly me, it was a trick question. I should have known better than to
fall into that trap. See the responses you got from other people already
and go check out the archives.

No even better, here's a link:
http://www.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/stable_api_nonsense.txt

If you still wish to argue for this, please take me off the CC list.
To be honest, I don't think anyone on linux-kernel is interested in yet
another round of this one.

Cheers,
Jes

Re: Linux drivers management

[Christoph Hellwig]

could you please shut up as non-contributor and parasite?

Re: Linux drivers management

[Denis Vlasenko]

On Monday 06 February 2006 18:56, Christoph Hellwig wrote:
> could you please shut up as non-contributor and parasite?

I don't like your tone at all. In case you will suggest me
to shut up too, I would not.
--
vda

Re: Linux drivers management

[Christoph Hellwig]

On Tue, Feb 07, 2006 at 01:36:06PM +0200, Denis Vlasenko wrote:
> On Monday 06 February 2006 18:56, Christoph Hellwig wrote:
> > could you please shut up as non-contributor and parasite?
> 
> I don't like your tone at all. In case you will suggest me
> to shut up too, I would not.

Not that I'd care too much when, but I wouldn't initially say that to you.
You've been doing some useful work, while David has not contributed anything
while abusing at least the -fsdevel list with stupid question to help
implementing his companies probably illegal filesystem.  He's the last
one who should speak up on lkml with any questions or suggestions.

Re: Linux drivers management

[Greg KH]

On Mon, Feb 06, 2006 at 05:45:59PM +0800, David Chow wrote:
> Is there any work in Linux undergoing to separate Linux drivers and the 
> the main kernel, and manage drivers using a package management system 
> that only manages kernel drivers and modules? If this can be done, the 
> kernel maintenance can be simple, and will end-up with a more stable 
> (less frequent changed) kernel API for drivers, also make every 
> developers of drivers happy.

The separation of drivers from the core kernel has nothing to do with
the stability of the in-kernel api.  To think otherwise is foolish, and
does not show an understanding of the current kernel apis.  See the
archives for all of the times this has come up previously.

> Would like to see that happens .

Feel free to submit patches to do so, if it is something you want to do.
Otherwise, telling other people to do something will not achieve
anything.

good luck,

greg k-h

Re: Linux drivers management

[Jim Crilly]

On 02/06/06 05:45:59PM +0800, David Chow wrote:
> Dear maintainers,
> 
> Is there any work in Linux undergoing to separate Linux drivers and the 
> the main kernel, and manage drivers using a package management system 
> that only manages kernel drivers and modules? If this can be done, the 
> kernel maintenance can be simple, and will end-up with a more stable 
> (less frequent changed) kernel API for drivers, also make every 
> developers of drivers happy.
> 
> Would like to see that happens .
> 


> regards,
> David Chow

Debian includes a tool call module-assistant that allows one to download,
compile and install the 3rd party modules that they package pretty
painlessly. But it obviously doesn't include the drivers already in the
kernel since they're included in the kernel packages.

Jim.