[ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[richardj_moore]

We've just release version 0.6 of Generalised Kernel Hooks Interface (GKHI)
see the IBM Linux Technology Centre's web page DProbes link:
http://oss.software.ibm.com/developerworks/opensource/linux

Some folks expressed an interest in this type of facility recently in
discussions concerning making call-backs from the kernel to kernel modules.

Here's the abstract for this facility. With this intend to modularise our
RAS offerings, in particular DProbes, so that they can be applied
dynamically without having to be carried as excess baggage.

Abstract:
Generalised Kernel Hooks Interface (GKHI) is a generalised facility for
placing hooks or exits in arbitrary kernel locations. It enables many
kernel enhancements, which are  otherwise self-contained, to become
loadable kernel modules and retain a substantial degree of independence
from the kernel source. This affords advantages for maintenance and
co-existence with other kernel enhancements. The hook interface allows
multiple kernel modules to register their exits for a given hook, in order
to receive control at that hook location. Multiple hooks may be defined
within the kernel and a singe kernel module may register exits to use
multiple hooks.  When hook exits register they may specify co-existence
criteria. Hooks may be placed in kernel modules as well as the kernel
itself with the proviso that the modules with hooks are loaded before the
gkhi hook interfacing module. A hook exit receives control as if called
from the code in which the hook is located. Parameters may be passed to a
hook exit and may be modified by an exit. For more information down-load
the tarball.

Note: GHKI is in late beta test - we currently do not support SMP, that
will occur soon. We also plan to support dynamic hook definition as little
later on so that kernel modules may dynamically register hooks for other
kernel modules to use.


Richard Moore -  RAS Project Lead - Linux Technology Centre (PISC).

http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd,  MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Michael Rothwell]

Sounds great; unfortunately, the core group has spoken out against a
modular kernel.

Perhaps IBM should get together with SGI, HP and other interested
parties and start an Advanced Linux Kernel Project. Then they can run
off and make their scalable, modular, enterprise kernel and the Linus
Version can always merge back in features from it.

-M

richardj_moore@uk.ibm.com wrote:
> 
> We've just release version 0.6 of Generalised Kernel Hooks Interface (GKHI)
> see the IBM Linux Technology Centre's web page DProbes link:
> http://oss.software.ibm.com/developerworks/opensource/linux
> 
> Some folks expressed an interest in this type of facility recently in
> discussions concerning making call-backs from the kernel to kernel modules.
> 
> Here's the abstract for this facility. With this intend to modularise our
> RAS offerings, in particular DProbes, so that they can be applied
> dynamically without having to be carried as excess baggage.
> 
> Abstract:
> Generalised Kernel Hooks Interface (GKHI) is a generalised facility for
> placing hooks or exits in arbitrary kernel locations. It enables many
> kernel enhancements, which are  otherwise self-contained, to become
> loadable kernel modules and retain a substantial degree of independence
> from the kernel source. This affords advantages for maintenance and
> co-existence with other kernel enhancements. The hook interface allows
> multiple kernel modules to register their exits for a given hook, in order
> to receive control at that hook location. Multiple hooks may be defined
> within the kernel and a singe kernel module may register exits to use
> multiple hooks.  When hook exits register they may specify co-existence
> criteria. Hooks may be placed in kernel modules as well as the kernel
> itself with the proviso that the modules with hooks are loaded before the
> gkhi hook interfacing module. A hook exit receives control as if called
> from the code in which the hook is located. Parameters may be passed to a
> hook exit and may be modified by an exit. For more information down-load
> the tarball.
> 
> Note: GHKI is in late beta test - we currently do not support SMP, that
> will occur soon. We also plan to support dynamic hook definition as little
> later on so that kernel modules may dynamically register hooks for other
> kernel modules to use.
> 
> Richard Moore -  RAS Project Lead - Linux Technology Centre (PISC).
> 
> http://oss.software.ibm.com/developerworks/opensource/linux
> Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
> IBM UK Ltd,  MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> Please read the FAQ at http://www.tux.org/lkml/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Christoph Rohland]

Hi Michael,

On Wed, 08 Nov 2000, Michael Rothwell wrote:
> Sounds great; unfortunately, the core group has spoken out against a
> modular kernel.
> 
> Perhaps IBM should get together with SGI, HP and other interested
> parties and start an Advanced Linux Kernel Project. Then they can
> run off and make their scalable, modular, enterprise kernel and the
> Linus Version can always merge back in features from it.

*Are you crazy?* =:-0 

Proposing proprietary kernel extensions to establish an enterprise
kernel? No thanks!

Greetings
		Christoph

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Larry McVoy]

On Thu, Nov 09, 2000 at 08:44:11AM +0100, Christoph Rohland wrote:
> Hi Michael,
> 
> On Wed, 08 Nov 2000, Michael Rothwell wrote:
> > Sounds great; unfortunately, the core group has spoken out against a
> > modular kernel.
> > 
> > Perhaps IBM should get together with SGI, HP and other interested
> > parties and start an Advanced Linux Kernel Project. Then they can
> > run off and make their scalable, modular, enterprise kernel and the
> > Linus Version can always merge back in features from it.
> 
> *Are you crazy?* =:-0 
> 
> Proposing proprietary kernel extensions to establish an enterprise
> kernel? No thanks!

Actually, I think this idea is a good one.  I'm a big opponent of all the
big iron feature bloat getting into the kernel, and if SGI et al want to
go off and do their own thing, that's fine with me.  As long as Linus 
continues in his current role, I doubt much of anything that the big iron
boys do will really make it back into the generic kernel.  Linus is really
smart about that stuff, are least it seems so to me; he seems to be well
aware that 99.9999% of the hardware in the world isn't big iron and never
will be, so something approximating 99% of the effort should be going towards
the common platforms, not the uncommon ones.
-- 
---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Andre Hedrick]

Second or Third here!!!

TRG plans to create and publish a native RING 0 kernel and packages.
This may end up as a bolt on ./arch or something.
Not everyone in the world needs a SUPERCHARGED, FUEL-INJECTED, ALCOHOL,
FIRE-BREATHING kernel, but some do!

Andre Hedrick
CTO Timpanogas Research Group
EVP Linux Development, TRG
Linux ATA Development


On Wed, 8 Nov 2000, Larry McVoy wrote:

> On Thu, Nov 09, 2000 at 08:44:11AM +0100, Christoph Rohland wrote:
> > Hi Michael,
> > 
> > On Wed, 08 Nov 2000, Michael Rothwell wrote:
> > > Sounds great; unfortunately, the core group has spoken out against a
> > > modular kernel.
> > > 
> > > Perhaps IBM should get together with SGI, HP and other interested
> > > parties and start an Advanced Linux Kernel Project. Then they can
> > > run off and make their scalable, modular, enterprise kernel and the
> > > Linus Version can always merge back in features from it.
> > 
> > *Are you crazy?* =:-0 
> > 
> > Proposing proprietary kernel extensions to establish an enterprise
> > kernel? No thanks!
> 
> Actually, I think this idea is a good one.  I'm a big opponent of all the
> big iron feature bloat getting into the kernel, and if SGI et al want to
> go off and do their own thing, that's fine with me.  As long as Linus 
> continues in his current role, I doubt much of anything that the big iron
> boys do will really make it back into the generic kernel.  Linus is really
> smart about that stuff, are least it seems so to me; he seems to be well
> aware that 99.9999% of the hardware in the world isn't big iron and never
> will be, so something approximating 99% of the effort should be going towards
> the common platforms, not the uncommon ones.
> -- 
> ---
> Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> Please read the FAQ at http://www.tux.org/lkml/
> 


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Christoph Rohland]

Hi Larry,

On Wed, 8 Nov 2000, Larry McVoy wrote:
> On Thu, Nov 09, 2000 at 08:44:11AM +0100, Christoph Rohland wrote:
>> *Are you crazy?* =:-0 
>> 
>> Proposing proprietary kernel extensions to establish an enterprise
>> kernel? No thanks!
> 
> Actually, I think this idea is a good one.  I'm a big opponent of
> all the big iron feature bloat getting into the kernel, and if SGI
> et al want to go off and do their own thing, that's fine with me.
> As long as Linus continues in his current role, I doubt much of
> anything that the big iron boys do will really make it back into the
> generic kernel.  Linus is really smart about that stuff, are least
> it seems so to me; he seems to be well aware that 99.9999% of the
> hardware in the world isn't big iron and never will be, so something
> approximating 99% of the effort should be going towards the common
> platforms, not the uncommon ones.

If we would not allow binary only modules I would not have such a big
problem with that...

I understand that the one size fits all approach has some limitations
if you want to run on PDAs up to big iron. But a framework to overload
core kernel functions with modules smells a lot of binary only, closed
source, vendor specific Linux on high end machines. 

And then I don't see the value of Linux anymore.

Greetings
		Christoph
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Michael Rothwell]

Christoph Rohland wrote:
> If we would not allow binary only modules I would not have such a big
> problem with that...

I'm not sure how you would do that.
 
> I understand that the one size fits all approach has some limitations
> if you want to run on PDAs up to big iron. But a framework to overload
> core kernel functions with modules smells a lot of binary only, closed
> source, vendor specific Linux on high end machines.

Since Linux is GPL, how would you stop this?
 
> And then I don't see the value of Linux anymore.

Same as before -- freedom and low cost. The primary advantae of Linux
over other OSes is the GPL. 

I think and Advanced Linux Kernel PRoject would be a good idea for a
number of reasons. It would give "Enterprise" users their own special
kernel, just like the microcontroller and real-time guys have. It would
also provide a parallel development track for Linux that could provide
real competition and value to the Linus-version kernel. The "Enterprise"
machines that IBM, HP and SGI would target aren't all S/390s; there
would be significant overlap of their low end with Linus' high end, I
think. Like 8+-way SMP servers.

-M
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Lars Marowsky-Bree]

On 2000-11-09T07:20:27,
   Michael Rothwell <rothwell@holly-springs.nc.us> said:

> > I understand that the one size fits all approach has some limitations
> > if you want to run on PDAs up to big iron. But a framework to overload
> > core kernel functions with modules smells a lot of binary only, closed
> > source, vendor specific Linux on high end machines.
> 
> Since Linux is GPL, how would you stop this?

Christoph / SAP is in a rather good position to stop that being supported by
vendors...

> Same as before -- freedom and low cost. The primary advantae of Linux
> over other OSes is the GPL. 

And that is why that has to govern the kernel and its modules as far as
possible.

Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>
    Development HA

-- 
Perfection is our goal, excellence will be tolerated. -- J. Yahl

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Alexander Viro]

On Thu, 9 Nov 2000, Michael Rothwell wrote:

> Same as before -- freedom and low cost. The primary advantae of Linux
> over other OSes is the GPL. 

Now, that's more than slightly insulting...

The problem with the hooks et.al. is very simple - they promote every
bloody implementation detail to exposed API. Sorry, but... See Figure 1.
It won't fly.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Michael Rothwell]

Alexander Viro wrote:
> 
> On Thu, 9 Nov 2000, Michael Rothwell wrote:
> 
> > Same as before -- freedom and low cost. The primary advantae of Linux
> > over other OSes is the GPL.
> 
> Now, that's more than slightly insulting...

Well, it wasn't meant to be. I imagine RMS would make the same type of
statement -- Linux is libre, therefore superior. There's a number of
OSes that have advantages of Linux in some area; Solaris can use more
processors; QNX is real-time, smaller and still posix; Windows has
better application support (i.e., more of them); MacOS has better color
and font management. But, Linux is free. Let's say for a moment that
Linux was exactly the same as Solaris, technically. Linux would be
superior because it is licensed under the GPL, and is free; whereas
Solaris would not be.

> The problem with the hooks et.al. is very simple - they promote every
> bloody implementation detail to exposed API. Sorry, but... See Figure 1.
> It won't fly.

Figure 1?

-M
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Alexander Viro]

On Thu, 9 Nov 2000, Michael Rothwell wrote:

> Alexander Viro wrote:
> > 
> > On Thu, 9 Nov 2000, Michael Rothwell wrote:
> > 
> > > Same as before -- freedom and low cost. The primary advantae of Linux
> > > over other OSes is the GPL.
> > 
> > Now, that's more than slightly insulting...
> 
> Well, it wasn't meant to be. I imagine RMS would make the same type of
> statement -- Linux is libre, therefore superior. There's a number of

<shrug> RMS had repeatedly demonstrated what he's worth as a designer
and programmer. Way below zero. You may like or dislike his ideology,
but when it comes to technical stuff... Not funny.

> OSes that have advantages of Linux in some area; Solaris can use more
> processors; QNX is real-time, smaller and still posix; Windows has
> better application support (i.e., more of them); MacOS has better color
> and font management. But, Linux is free. Let's say for a moment that
> Linux was exactly the same as Solaris, technically. Linux would be

You mean, bloated tasteless parody on UNIX? Thanks, but no thanks.

> superior because it is licensed under the GPL, and is free; whereas
> Solaris would not be.

Small solace it would be.

> > The problem with the hooks et.al. is very simple - they promote every
> > bloody implementation detail to exposed API. Sorry, but... See Figure 1.
> > It won't fly.
> 
> Figure 1?

Use search engine. On google "See Figure 1" brings the thing in the first
5 hits.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Michael Rothwell]

Alexander Viro wrote:

> > Figure 1?
> 
> Use search engine. On google "See Figure 1" brings the thing in the first
> 5 hits.

http://www.google.com/search?q=See+Figure+1&btnG=Google+Search
->
http://spiffy.cso.uiuc.edu/~kline/Stuff/see-figure-1.html
->
http://spiffy.cso.uiuc.edu/~kline/Stuff/f-you.gif

...

http://www.google.com/search?q=See+Figure+1&btnG=Google+Search
->
http://www.physics.ohio-state.edu/~bcd/humor/figure1.html

... How utterly typical of you, Viro. 

-M
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Mike Coleman]

Alexander Viro <viro@math.psu.edu> writes:
> <shrug> RMS had repeatedly demonstrated what he's worth as a designer
> and programmer. Way below zero. You may like or dislike his ideology,
> but when it comes to technical stuff... Not funny.

Huh?

<annoying valspeak tone>

*Hello*?  GNU gcc?  GNU emacs?  Way below zero?  *Hello*?!?

</annoying valspeak tone>


-- 
[O]ne of the features of the Internet [...] is that small groups of people can
greatly disturb large organizations.  --Charles C. Mann
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Alexander Viro]

On 9 Nov 2000, Mike Coleman wrote:

> Alexander Viro <viro@math.psu.edu> writes:
> > <shrug> RMS had repeatedly demonstrated what he's worth as a designer
> > and programmer. Way below zero. You may like or dislike his ideology,
> > but when it comes to technical stuff... Not funny.
> 
> Huh?
> 
> <annoying valspeak tone>
> 
> *Hello*?  GNU gcc?  GNU emacs?  Way below zero?  *Hello*?!?
> 
> </annoying valspeak tone>

Yes. GNU emacs. Bloated and ugly like hell. gcc. Codebase is anything but
pretty. Your point being? And no, it's not trolling. RMS really got no
taste - read his postings on _technical_ GNU lists and see yourself.

Crap can work. Given enough thrust pigs will fly, but it's not necessary a
good idea.

$DEITY witness, we've got enough ugly code, but compared to the average
level of GNU codebase kernel looks wonderful. Sorry. I think that I've
spent enough time dealing with both to be able to compare.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Martin Dalecki]

Alexander Viro wrote:
> 
> On 9 Nov 2000, Mike Coleman wrote:
> 
> > Alexander Viro <viro@math.psu.edu> writes:
> > > <shrug> RMS had repeatedly demonstrated what he's worth as a designer
> > > and programmer. Way below zero. You may like or dislike his ideology,
> > > but when it comes to technical stuff... Not funny.
> >
> > Huh?
> >
> > <annoying valspeak tone>
> >
> > *Hello*?  GNU gcc?  GNU emacs?  Way below zero?  *Hello*?!?
> >
> > </annoying valspeak tone>
> 
> Yes. GNU emacs. Bloated and ugly like hell. gcc. Codebase is anything but
> pretty. Your point being? And no, it's not trolling. RMS really got no
> taste - read his postings on _technical_ GNU lists and see yourself.

Actually emacs is complete crap int terms of both ergonomy and coding.
But the current gcc code base doesn't contain much from RMS anylonger. 
The "beauty" of lisp coded in C it is...
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Marco Colombo]

On Thu, 9 Nov 2000, Michael Rothwell wrote:

> Alexander Viro wrote:
> > 
> > On Thu, 9 Nov 2000, Michael Rothwell wrote:
> > 
> > > Same as before -- freedom and low cost. The primary advantae of Linux
> > > over other OSes is the GPL.
> > 
> > Now, that's more than slightly insulting...
> 
> Well, it wasn't meant to be. I imagine RMS would make the same type of
> statement -- Linux is libre, therefore superior. There's a number of
> OSes that have advantages of Linux in some area; Solaris can use more
> processors; QNX is real-time, smaller and still posix; Windows has
> better application support (i.e., more of them); MacOS has better color
> and font management. But, Linux is free. Let's say for a moment that
> Linux was exactly the same as Solaris, technically. Linux would be
> superior because it is licensed under the GPL, and is free; whereas
> Solaris would not be.

<lurker off>
Sorry, I couldn't resist.

1) "Solaris running on more processors"? Think of Beowulf clusters.
   Strangely enough, people running them choose Linux as the kernel.
2) "QNX is real-time", true. Linux is not. Please don't compare apples with
   oranges.
3) "Windows has more apps"? True. Is it a kernel property of any kind? No.
4) "MacOS has better color and font management" this is funny as it speaks
   for itself. I'd also add "MacOS is usually housed in a better-looking
   case". Luckily enough, under Linux color and font management is NOT a
   kernel job. No more than the external design of the case, I mean.

And I really hope Linux will *never* be exactly the same as Solaris,
technically.
</lurker on>

> > The problem with the hooks et.al. is very simple - they promote every
> > bloody implementation detail to exposed API. Sorry, but... See Figure 1.
> > It won't fly.
> 
> Figure 1?

>From "The design and implementation of the Perfect OS 1.0", yet to be
published. The good thing about this book is that there will never be
a second edition. Of course the only release of Perfect OS will be 1.0!
B-) B-) B-) B-)

> 
> -M
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> Please read the FAQ at http://www.tux.org/lkml/
> 

.TM.
-- 
      ____/  ____/   /
     /      /       /			Marco Colombo
    ___/  ___  /   /		      Technical Manager
   /          /   /			 ESI s.r.l.
 _____/ _____/  _/		       Colombo@ESI.it


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Christoph Rohland]

Hi Michael,

On Thu, 09 Nov 2000, Michael Rothwell wrote:
> Christoph Rohland wrote:
>> And then I don't see the value of Linux anymore.
> 
> Same as before -- freedom and low cost. The primary advantae of
> Linux over other OSes is the GPL.

And you would loose exactly these two points for high end servers.

Greetings
		Christoph

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Tigran Aivazian]

On Wed, 8 Nov 2000, Larry McVoy wrote:
> As long as Linus continues in his current role, I doubt much of
> anything that the big iron boys do will really make it back into the
> generic kernel.

That is great, thank you. At least I know now someone on this planet who
agrees with me! Everyone (regardless of whether he has seen commercial
UNIX source or not, but it pains most to hear from those who have) seems
to be sceptical when I say exactly the above. They think "how could that
possibly be -- the "big iron Unixen" had such a large investment of real
money to make them full of "enterprise-features", surely there must be
something useful for Linux to learn from". To which I reply -- "most
(probably 99-100%) Linux kernel hackers have access to the source code of
most (probably 40%-90% depending on their lack) flavours of commercial
UNIX and would be quite happy to "steal" code from it (as long as they,
like myself agree with RMS philosophy/morals) but the reality is -- there
is _nothing_ to steal from it". Commercial UNIXen are just useless -- we
_have_ to rewrite everything from scratch, not because we can't steal (or
think it immoral) but because there is nothing interesting to steal left.
The free stuff is actually better and technically superior to that which
is not free.

To contradict to myself (just a tiny bit!) I think it would still be
useful for the public in general (by "public in general" I meant those
clueless individuals who write the laws men have to obey) to recognize the
above fact and let the hackers be no longer shy about it, i.e. be able to
freely discuss any information they have (or had) access to, so that, in
the unlikely case that there is something useful to "steal" from it, they
can take it freely and put it into Linux.

Regards,
Tigran

PS. The very fact I can say the above and still (hope to) have a job
afterwards is a sign of much progress towards the better end :)

PPS. Better is the end of a thing than the beginning thereof: and the
patient in spirit is better than the proud in spirit. (Eccl 7:8)

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Ingo Molnar]

On Wed, 8 Nov 2000, Larry McVoy wrote:

> smart about that stuff, are least it seems so to me; he seems to be
> well aware that 99.9999% of the hardware in the world isn't big iron
> and never will be, so something approximating 99% of the effort should
> be going towards the common platforms, not the uncommon ones.

yep, this is true. Still Linux appears to perform remarkably well on
so-called 'big iron'.

IMO it's a big misconception that 'big iron is different'. Yes, patches
and bad design can make source trees very different. But IMO big iron is
not more different from a normal PIII workstation than a PDA is different.
We doing a bad job if we cannot support them all - or at least we must
always be able to keep clean interfaces so keeping a forked sub-project
for a less known or less understood feature is easy and straightforward.
In fact i believe a PDA is much harder to do right than high-end systems.
Making something faster, given endless resources, is almost always easy.
But maximizing performance on a fundamentally and intentionally limited
platform is much less trivial and takes alot of clout.

in the 2.4 kernel we have all the features that is needed for 'enterprise
scalability', in fact i believe we have some scalability features (eg. big
reader spinlocks) that are not available on other systems.

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Christoph Rohland]

Hi Ingo,

On Thu, 9 Nov 2000, Ingo Molnar wrote:
> 
> On Wed, 8 Nov 2000, Larry McVoy wrote:
> 
>> smart about that stuff, are least it seems so to me; he seems to be
>> well aware that 99.9999% of the hardware in the world isn't big
>> iron and never will be, so something approximating 99% of the
>> effort should be going towards the common platforms, not the
>> uncommon ones.
> 
> yep, this is true. Still Linux appears to perform remarkably well on
> so-called 'big iron'.

Thanks Ingo, I agree to this and also agree that we should try to be
able to run (mostly) everything from the same code base and I think
that's Linux does a great job on this.

Having a seperated code base for 0.0001% of mission critical servers
will lead to very bad availability or very high development cost at
least. In the worst (and not so unprobable) case it will lead to a lot
of games with licenses and 'intellectual property'. This would mean
that Linux fails to deliver on its promises IMHO.

Greetings
		Christoph
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Theodore Y. Ts'o]

   Date: 	Wed, 08 Nov 2000 16:35:33 -0500
   From: Michael Rothwell <rothwell@holly-springs.nc.us>

   Sounds great; unfortunately, the core group has spoken out against a
   modular kernel.

This is true; that's because a modular kernel means that interfaces have
to be frozen in time, usually forever.  This makes a lot of improvements
impossible, and over time there is more and more crud added to simply
act as "impedance matching" between incompatible/legacy interfaces.

However, that doesn't mean that the GKHI is *automatically* bad.  It all
depends on how you use it.  Just as with kernel modules, where it's
darned useful for distributions so they don't have to build custom
kernels for each installation (but source is included for all modules),
the GKHI could be used in a similar way. 

The issue will be with what hooks are allowed to be exported via
the GKHI; this defines the interfaces.  Also, it's important to note
which interfaces are and aren't guaranteed to be stable.  If the
interfaces aren't guaranteed to be stable, then incompatibilities and
keeping up with the latest version are a problem for the provider of the
binary module, not of the Linux Kernel Development Community.

						- Ted
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Matti Aarnio]

  I have been wondering what all of the furor has been about...

	Initially I thought that it is "a way to load in a module which
	defines its own syscalls, etc.." and/or "we want to sell binary
	images which can activate some hooks" but having just read the
	GKHI README, that thing is far away from its intentions.
	(Well, it doesn't preclude those, but neither it mentions them
	 as objectives.  And giving a license stating use of GNU GPL
	 also doesn't quite fit "proprietary binary hook" image..)

On Wed, Nov 08, 2000 at 04:35:33PM -0500, Michael Rothwell wrote:
> Sounds great; unfortunately, the core group has spoken out against a
> modular kernel.

	Really ?

$ /sbin/lsmod 
Module                  Size  Used by
parport_pc             23184   1 (autoclean)
lp                      5072   0 (autoclean) (unused)
parport                30048   1 (autoclean) [parport_pc lp]
8021q                  10032   2
3c59x                  24304   2 (autoclean)
ipv6                  152816  -1 (autoclean)
autofs                 11536   1 (autoclean)
usb-uhci               23408   0 (autoclean) (unused)
usbcore                49504   1 (autoclean) [usb-uhci]
es1371                 29920   0
ac97_codec              7824   0 [es1371]
soundcore               4336   4 [es1371]

> -M
> 
> richardj_moore@uk.ibm.com wrote:
> > 
> > We've just release version 0.6 of Generalised Kernel Hooks Interface (GKHI)
> > see the IBM Linux Technology Centre's web page DProbes link:
> > http://oss.software.ibm.com/developerworks/opensource/linux

... (reordered, cut away..)

> > Here's the abstract for this facility. With this intend to modularise our
> > RAS offerings, in particular DProbes, so that they can be applied
> > dynamically without having to be carried as excess baggage.

	Richard,

	Please educate me, what does "our RAS offerings" mean here ?
	(I didn't find "RAS" at your signature-URL site, but I didn't
	 poke around very much..)

	I do know that when IBM suits speak with phrases like that,
	they are selling me something which costs $$$.

	Which definitely gives proprietary, binary only, hook image...
	But GKHI, and DProbes are neither. Thus I am confused, but can
	understand the furor...

> > Some folks expressed an interest in this type of facility recently in
> > discussions concerning making call-backs from the kernel to kernel modules.

	Indeed,  one such mechanism could be a way to register IOCTL
	call chains, which now (for sockets) are quite ugly.
	Lots and lots of subsystems do ioctl()s via /proc/ objects
	just because other methods are way too messy.

	[ ioctl's go via the protocol family of the control socket to
	  family-specific subset, but then the "fun" begins for things
	  which aren't quite of any specific protocol family -- see
	  DLCI support hooks at  ipv4,  and bridge ioctls at both ipv4
	  and at packet.

	  Grep the kernel source for "_hook", and you see a lot of
	  things..  Mostly varying mouses, and bridging, it seems.
	  Netfilter calls its managed coherent interface "hook", but
	  it is way better. ]

	Also the bridging system is less than desirable looking with
	its pervasive hooks, but that can be solved by making layer2
	devices fully stackable.  (Something for 2.5)



> > Richard Moore -  RAS Project Lead - Linux Technology Centre (PISC).
> > http://oss.software.ibm.com/developerworks/opensource/linux
> > Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
> > IBM UK Ltd,  MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK

/Matti Aarnio
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Michael Rothwell]

Matti Aarnio wrote:
> On Wed, Nov 08, 2000 at 04:35:33PM -0500, Michael Rothwell wrote:
> > Sounds great; unfortunately, the core group has spoken out against a
> > modular kernel.
> 
>         Really ?
> 
> $ /sbin/lsmod
> Module                  Size  Used by
> [...]
> soundcore               4336   4 [es1371]


Really. That the kernal has loadable modules does not mean that it is
modular.

-M
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Advanced Linux Kernel/Enterprise Linux Kernel

[Josue Emmanuel Amaro]

[-- Attachment #1: Type: text/plain, Size: 1390 bytes --]

This subject came up in the Generalized Kernel Hooks Interface thread, since it
is an area of interest to me I wanted to continue that conversation.

While I do not think it would be productive to enter a discussion whether there
is a need to fork the kernel to add features that would be beneficial to
mission/business critical applications, I am curious as to what are the features
that people consider important to have.  By mission critical I mean systems that
if not functional bring a business to a halt, while by business critical I mean
systems that affect a division of a business.

Another problem is how people define Enterprise Systems.  Many base it on the
definitions that go back to S390 systems, others in the context of the 24/7
nature of the internet.  That would also be a healthy discussion to have.

At Oracle we are primarily interested on I/O subsystem features and memory
management.  (For anyone that knows anything about Oracle this should not come
as a surprise, although I am pretty sure that any database vendor/developer
would be interested on those features as well.)

Regards,

--
=======================================================================
  Josue Emmanuel Amaro                         Josue.Amaro@oracle.com
  Linux Products Manager
  Intel and Linux Technologies Group
=======================================================================


[-- Attachment #2: Card for Josue Emmanuel Amaro --]
[-- Type: text/x-vcard, Size: 390 bytes --]

begin:vcard 
n:Amaro;Josue Emmanuel
tel;cell:650-245-5131
tel;fax:650-413-0167
tel;work:650-506-1239
x-mozilla-html:FALSE
url:http://www.oracle.com
org:Intel and Linux Technologies
version:2.1
email;internet:Josue.Amaro@oracle.com
title:Sr.Product Manager - Linux
adr;quoted-printable:;;500 Oracle Parkway=0D=0AMS1ip4;Redwood Shores;CA;94065;United States
fn:Josue Emmanuel Amaro
end:vcard

Re: Advanced Linux Kernel/Enterprise Linux Kernel

[Lars Marowsky-Bree]

On 2000-11-13T13:56:16,
   Josue Emmanuel Amaro <Josue.Amaro@oracle.com> said:

Good morning Josue,

I hope your certification matrix hasn't driven you mad yet ;-)

> While I do not think it would be productive to enter a discussion whether
> there is a need to fork the kernel to add features that would be beneficial
> to mission/business critical applications, I am curious as to what are the
> features that people consider important to have.

This is in fact the valuable subpart of the discussion.

Working for SuSE on High Availability, especially in the "enterprise" segment:
Here, referring to systems running databases (mostly Oracle, surprise),
ERP-Systems, but also providing services (NFS, Samba, firewalls) in such an
environment.

I personally need features which allow me to keep on running, shut down as
gracefully as possible if an error occurs, and if an error occured, diagnose
it out in the field.

This means: ECC memory, hotpluggable everything, proper error handling and
reporting in the kernel. Yes, christmas and easter do occur on the same day in
the real world, unfortunately.

This can best be summarised as "robustness".

If an error occured, I need to be able to fully diagnose it without having to
reproduce it - no, I do not wish to reproduce the error by crashing my
critical server on purpose, nor is "The error appears to have gone away, we
have no clue what it was" an acceptable answer. (kdb, LKCD, Oopsing to the
network etc: And they must be part of the default kernel as far as possible,
so they stay in sync and get widespread testing)

But also scalability: 2TB is a problem for me in some cases, 32bit just don't
cut it all the time - but I need to circumvent the storage problem even on a
32bit system. And adding disks to the system while running is desireable.

Cluster awareness, again mostly referring to storage: Yes, there is more than
one system accessing my SCSI bus, my FCAL RAID, and the error handling should
be architected in a way that they do not start reset wars.

The LVM should safeguard against multiple nodes changing the metadata. (Ok,
this can be solved in userspace too) LVM must be transactional, so a crash on
a node doesn't corrupt the data.

Basically, the talks in Miami (The Second Annual Linux Storage Management
Workshop) gave a great overview of everything I need.

And: I need all of this as Open Source. Period. No binary kernel modules do me
any good and I will pointedly ignore them.

Oh, and by the way - if any hot kernel hacker, not yet working on this full
time feels inspired to make this happen, contact me. Or any other Linux
company, as long as the job gets done. We'll be glad to make you a fulltime
kernel slave^Whacker! ;-)

> Another problem is how people define Enterprise Systems.  Many base it on the
> definitions that go back to S390 systems, others in the context of the 24/7
> nature of the internet.  That would also be a healthy discussion to have.
           _
24/7 * 99.99% mission/business critical services with "medium to high" load.

Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>
    Development HA

-- 
Perfection is our goal, excellence will be tolerated. -- J. Yahl

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Advanced Linux Kernel/Enterprise Linux Kernel

[lamont]

if you look at the kstat structure under solaris, there's a lot of info
there that'd be good to be able to pull out of the linux kernel.  that
would slow down the kernel a little, lead to some 'bloat' that linus
abhors and such, but its good to have that information for monitoring and
debugging problems.  it'd also be nice to have hooks built in to monitor
errors in the disk subsystem and ideally warn of impending failures as
much as possible -- that might be better done in userspace.  and pretty
much you want all error messages from different subsystems to be monitored
and appropriate action taken.  ideally all error messages from the kernel
and device drivers should be standardized and HA software can then monitor
them and take appropriate actions when it sees one that indicates
failure.  sun is currently in the process of documenting all the kernel
error messages from what i understand.  those are the kinds of things that
give IT managers and sysadmins warm fuzzy feelings about solaris.

On Mon, 13 Nov 2000, Josue Emmanuel Amaro wrote:
> This subject came up in the Generalized Kernel Hooks Interface thread, since it
> is an area of interest to me I wanted to continue that conversation.
> 
> While I do not think it would be productive to enter a discussion whether there
> is a need to fork the kernel to add features that would be beneficial to
> mission/business critical applications, I am curious as to what are the features
> that people consider important to have.  By mission critical I mean systems that
> if not functional bring a business to a halt, while by business critical I mean
> systems that affect a division of a business.
> 
> Another problem is how people define Enterprise Systems.  Many base it on the
> definitions that go back to S390 systems, others in the context of the 24/7
> nature of the internet.  That would also be a healthy discussion to have.
> 
> At Oracle we are primarily interested on I/O subsystem features and memory
> management.  (For anyone that knows anything about Oracle this should not come
> as a surprise, although I am pretty sure that any database vendor/developer
> would be interested on those features as well.)
> 
> Regards,
> 
> --
> =======================================================================
>   Josue Emmanuel Amaro                         Josue.Amaro@oracle.com
>   Linux Products Manager
>   Intel and Linux Technologies Group
> =======================================================================
> 
> 
> 

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[richardj_moore]

Let be clear about one thing: the GKHI make no statement about enabling
proprietary extensions and that's a common misconception. GKHI is intended
to make optional facilities easier to co-install and change. We designed it
for DProbes, and when modularised will remain a GPL opensource offering.

The only motivation for providing GKHI is to make the kernel more
acceptable to the enterprise customer, but allowing, for example, RAS
capabilities to be brough in easily and dynmaically. This type of customer
will not readily succome to on-the-fly kernel rebuilds to diagnose problems
that occur only in complex production environments.

If anything opens the door to proprietary extensions it's the loadable
kernel modules capability or perhaps the loose wording of the GPL which
doesn't catch loadable kernel modules, or whatever... Bottom line GKHI
really has no bearing on this.


Richard Moore -  RAS Project Lead - Linux Technology Centre (PISC).

http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd,  MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK


Christoph Rohland <cr@sap.com> on 09/11/2000 07:44:11

Please respond to Christoph Rohland <cr@sap.com>

To:   Michael Rothwell <rothwell@holly-springs.nc.us>
cc:   Richard J Moore/UK/IBM@IBMGB, linux-kernel@vger.kernel.org
Subject:  Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)




Hi Michael,

On Wed, 08 Nov 2000, Michael Rothwell wrote:
> Sounds great; unfortunately, the core group has spoken out against a
> modular kernel.
>
> Perhaps IBM should get together with SGI, HP and other interested
> parties and start an Advanced Linux Kernel Project. Then they can
> run off and make their scalable, modular, enterprise kernel and the
> Linus Version can always merge back in features from it.

*Are you crazy?* =:-0

Proposing proprietary kernel extensions to establish an enterprise
kernel? No thanks!

Greetings
          Christoph




-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Christoph Rohland]

Hi Richard,

On Thu, 9 Nov 2000, richardj moore wrote:
> Let be clear about one thing: the GKHI make no statement about
> enabling proprietary extensions and that's a common
> misconception. GKHI is intended to make optional facilities easier
> to co-install and change. We designed it for DProbes, and when
> modularised will remain a GPL opensource offering.

Yes, I understand that.

> The only motivation for providing GKHI is to make the kernel more
> acceptable to the enterprise customer, but allowing, for example,
> RAS capabilities to be brough in easily and dynmaically. This type
> of customer will not readily succome to on-the-fly kernel rebuilds
> to diagnose problems that occur only in complex production
> environments.

I know this problem pretty well.

> If anything opens the door to proprietary extensions it's the
> loadable kernel modules capability or perhaps the loose wording of
> the GPL which doesn't catch loadable kernel modules, or
> whatever... Bottom line GKHI really has no bearing on this.

Yes, and that's why I am opposing here: Technically you are right, but
proposing that enterprise Linux should go this way is inviting binary
only modules due to the lax handling of modules.

Please keep in mind: I did not react to your announcement but to the
proposal that the companies should jump on it to do a special
enterprise Linux. If we really need a special enterprise tree lets do
it without module tricks.

Greetings
		Christoph
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Michael Rothwell]

Christoph Rohland wrote:
> If we really need a special enterprise tree lets do
> it without module tricks.

Why? I think the IBM GKHI code would be of tremendous value. It would
make the kernel much more flexible, and for users, much more friendly.
No more patch-and-recompile to add a filesystem or whatever. There's no
reason to hamstring their efforts because of the possibility of binary
modules. The GPL allows that, right? So any developer of binary-only
extensions using the GKHI would not be breaking the license agreement, I
don't think. There's lots of binary modules right now -- VMWare, Aureal
sound card drivers, etc.

I understand and agree with your desire for full source for everything,
but I disagree that we should artificially limit people's ability to use
Linux to solve their problems.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Lars Marowsky-Bree]

On 2000-11-09T07:25:52,
   Michael Rothwell <rothwell@holly-springs.nc.us> said:

> Why? I think the IBM GKHI code would be of tremendous value. It would
> make the kernel much more flexible, and for users, much more friendly.
> No more patch-and-recompile to add a filesystem or whatever. There's no
> reason to hamstring their efforts because of the possibility of binary
> modules. The GPL allows that, right? So any developer of binary-only
> extensions using the GKHI would not be breaking the license agreement, I
> don't think. There's lots of binary modules right now -- VMWare, Aureal
> sound card drivers, etc.

And we already refuse to support those kernels - your point being?

Making this "commonplace" is a nightmare. Go away with that.

> I understand and agree with your desire for full source for everything,
> but I disagree that we should artificially limit people's ability to use
> Linux to solve their problems.

I want their solving of their problems not to create problems for me though.

Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>
    Development HA

-- 
Perfection is our goal, excellence will be tolerated. -- J. Yahl

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Michael Rothwell]

Lars Marowsky-Bree wrote:
> And we already refuse to support those kernels - your point being?

Who says you would support theirs? My point is, forks have been made in
the past and are useful for the people that use them, and prevent
"pollution" of the common kernel with hghly specialized modifications.
uCLinux and RTLinux are two examples.

> Making this "commonplace" is a nightmare. Go away with that.

It would be a third major fork (AFAIK), not a first, and not a
nightmare. Are RTLinux and uclinux nightmares? How much do they impact
your life?
 
> I want their solving of their problems not to create problems for me though.

How would it create problems for you? And as a separate question, why
should anyone care if it does? Part of the freedom guaranteed by the GPL
is the ability for anyone to fork a codebase for their own use. And
because it's all GPL code, with thoroughly divirsified copyright
assignment, they will be releasing any mods under GPL as well. The ones
the Linus likes, he can adapt and merge into the common kernel. If he
doesn't like them, he can ignore them.

Incidentally, I hear that a Utah company is going to position an older
Unix kernel with Linux compatibility added and possibly a GNU userspace
as "Enterprise Linux." Would you prefer that, or something actually
based on the Linux codebase?

-M
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Alan Cox]

> > Making this "commonplace" is a nightmare. Go away with that.
> 
> It would be a third major fork (AFAIK), not a first, and not a
> nightmare. Are RTLinux and uclinux nightmares? How much do they impact
> your life?

RTLinux is hardly a fork. UcLinux is a fork, it has its own mailing list, web
site and everything. Post 2.4 I'm still very interested in spending time merging
the 2.4 uc and the main tree. I think it can be done and they are doing it in
a way that leads logically to this.

To a lot of people the ucLinux is 2.0 and our MMU based boards run 2.2.18 and
this and that are different is a real pain


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Michael Rothwell]

Alan Cox wrote:

> RTLinux is hardly a fork. UcLinux is a fork, it has its own mailing list, web
> site and everything. Post 2.4 I'm still very interested in spending time merging
> the 2.4 uc and the main tree. I think it can be done and they are doing it in
> a way that leads logically to this.


And how would a hypothetical Advanced Linux Kernel Project be different?
Set aside the GKHI and the issue of binary-only hook modules; how would
an "enterprise" fork be any different than RT or UC? It'll go off,
change and add some things, and then perhaps be merged back in later. In
the meantime, developers who want to add "enterpriseness" to Linux will
have an outlet and won't have to simply gripe on this list anymore. And
users who want an "enterprise" kernel can get one.

-M
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Theodore Y. Ts'o]

   Date: 	Thu, 09 Nov 2000 08:43:14 -0500
   From: Michael Rothwell <rothwell@holly-springs.nc.us>

   And how would a hypothetical Advanced Linux Kernel Project be different?
   Set aside the GKHI and the issue of binary-only hook modules; how would
   an "enterprise" fork be any different than RT or UC? It'll go off,
   change and add some things, and then perhaps be merged back in later. In
   the meantime, developers who want to add "enterpriseness" to Linux will
   have an outlet and won't have to simply gripe on this list anymore. And
   users who want an "enterprise" kernel can get one.

Well, there are three possibilities about how it can end up.

1)  It will be loaded with so much crap that in fact it ends up being
less performant than the mainline Linux.  No problem, we can ignore it.

2)  It will be a thing of perfect beauty, in which every single change
is thoughtfully well-considered, and scales well to both the low end and
the high end.  No problem, it's easy to merge stuff back.

3) It will be a mixture, of some stuff which is good, and some such
which is crap, and it will be very difficult hard to merge back some of
the good stuff, since it has dependencies on the crap.  In the meantime,
mainline Linux will have continued moving forward, and it will be even
harder to merge the advanced features of Linux back into the forked
version of the kernel.


One of the big reasons why many of the big companies have been looking
at Linux is because Unix OS engineering is viewed as a cost center, not
as a profit center.  The moment you fork and make an "Advanced Linux
Kernel Project", a lot of the costs involved with maintaining such a
beast come back.  And sure, you can try to solve this problem by working
in a consoritum-like fashion with other Companies ---- just like OSF/1
and Monterrey tried to do.  

So there are some real costs associated with forking.  At the same time,
if these companies need to get product out the door quickly, short-term
forks can be good things.  But nothing in life is free, and it's in
*everybody's* best interest to resolve forks as soon as possible.
Otherwise, you end up losing a lot of the advantages of the Open Source
development model.

						- Ted
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Paul Jakma]

On Thu, 9 Nov 2000, Michael Rothwell wrote:

> Why? I think the IBM GKHI code would be of tremendous value. It would
> make the kernel much more flexible, and for users, much more friendly.
> No more patch-and-recompile to add a filesystem or whatever. There's no
> reason to hamstring their efforts because of the possibility of binary
> modules. The GPL allows that, right? 

no gpl definitely does not alow binary modules.

afaik linus allows binary modules in most cases.

> So any developer of binary-only
> extensions using the GKHI would not be breaking the license agreement, I
> don't think. There's lots of binary modules right now -- VMWare, Aureal
> sound card drivers, etc.
> 
> I understand and agree with your desire for full source for everything,
> but I disagree that we should artificially limit people's ability to use
> Linux to solve their problems.
> -

--paulj

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Michael Rothwell]

Paul Jakma wrote:
> 
> On Thu, 9 Nov 2000, Michael Rothwell wrote:
> 
> > Why? I think the IBM GKHI code would be of tremendous value. It would
> > make the kernel much more flexible, and for users, much more friendly.
> > No more patch-and-recompile to add a filesystem or whatever. There's no
> > reason to hamstring their efforts because of the possibility of binary
> > modules. The GPL allows that, right?
> 
> no gpl definitely does not alow binary modules.

Well, then, problem solved.

> afaik linus allows binary modules in most cases.

And since an "Advanced Linux Kernel Project" wouldn't be a Linus kernel,
what then? Would they have the same discretion as Linus? Would Linus'
exception apply to them?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Paul Jakma]

On Thu, 9 Nov 2000, Michael Rothwell wrote:

> Well, then, problem solved.
> 

:)

> > afaik linus allows binary modules in most cases.
> 
> And since an "Advanced Linux Kernel Project" wouldn't be a Linus kernel,
> what then? Would they have the same discretion as Linus? Would Linus'
> exception apply to them?

don't know. you'd have to ask him...

I actually think Linus has been too loose/vague on modules. The
official COPYING txt file in the tree contains an exception on linking
to the kernel using syscalls from linus and the GPL. nothing about
binary modules, and afaik the only statements he's ever made about
binary modules were off the cuff on l-k a long time (unless someone
knows a binary module whose vendor can show a written exception from
Linus et al). 

The result of all this is that we've had plenty of vendors ignoring
the GPL as it applies to linux and release binary modules all because
linus said on a mailling list that he doesn't mind too much. not a
very strong affirmation of the conditions under which linux is
licensed.

be nice if the binary module thing could be clarified by the copyright
holders.

--paulj

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Marco Colombo]

On Thu, 9 Nov 2000, Paul Jakma wrote:

> On Thu, 9 Nov 2000, Michael Rothwell wrote:
> 
> > Well, then, problem solved.
> > 
> 
> :)
> 
> > > afaik linus allows binary modules in most cases.
> > 
> > And since an "Advanced Linux Kernel Project" wouldn't be a Linus kernel,
> > what then? Would they have the same discretion as Linus? Would Linus'
> > exception apply to them?
> 
> don't know. you'd have to ask him...
> 
> I actually think Linus has been too loose/vague on modules. The
> official COPYING txt file in the tree contains an exception on linking
> to the kernel using syscalls from linus and the GPL. nothing about
> binary modules, and afaik the only statements he's ever made about
> binary modules were off the cuff on l-k a long time (unless someone
> knows a binary module whose vendor can show a written exception from
> Linus et al). 
> 
> The result of all this is that we've had plenty of vendors ignoring
> the GPL as it applies to linux and release binary modules all because
> linus said on a mailling list that he doesn't mind too much. not a
> very strong affirmation of the conditions under which linux is
> licensed.

Well, HW vendors may provide a binary module as a timid attempt to 
support Linux. A few have already understood that providing an Open Source
one is far a better attitude: they can *get* support for it from the
kernel community. They end up with a better driver, and they can even
learn something useful for their W98/NT/Sco/whatever drivers, too.
If they don't abuse of it (they are sicerely willing to "provide" something)
it's clearly a winning move.
Other vendors are just scared of the two words "Open Source" so they make
a little first step in releasing a binary only driver, which they are
more used to. I believe that sooner or later they'll realize the advantages
of the Open Source attitude, and they'll make the move.

A binary only file-system module is a completely different matter.
Legally, it may have the same "status" of a binary only driver. 
Technically, it's just another module. But it seems to me a much clearer
violation of GPL. If you want to hide the internals of your software,
you're not GPL-compatible (a driver is slightly different in that 
a HW company is probably worried about the internals of their HW).

> 
> be nice if the binary module thing could be clarified by the copyright
> holders.

Of course.

> 
> --paulj
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> Please read the FAQ at http://www.tux.org/lkml/
> 

.TM.
-- 
      ____/  ____/   /
     /      /       /			Marco Colombo
    ___/  ___  /   /		      Technical Manager
   /          /   /			 ESI s.r.l.
 _____/ _____/  _/		       Colombo@ESI.it

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Theodore Y. Ts'o]

   Date: 	Thu, 9 Nov 2000 13:39:04 +0000 (GMT)
   From: Paul Jakma <paulj@itg.ie>

   I actually think Linus has been too loose/vague on modules. The
   official COPYING txt file in the tree contains an exception on linking
   to the kernel using syscalls from linus and the GPL. nothing about
   binary modules, and afaik the only statements he's ever made about
   binary modules were off the cuff on l-k a long time (unless someone
   knows a binary module whose vendor can show a written exception from
   Linus et al). 

Actually, he's been quite specific.  It's ok to have binary modules as
long as they conform to the interface defined in /proc/ksyms.  


That being said, the real problem with the GKHI is that as Al said, it
does expose internal kernel interfaces --- and the Linux kernel
development community as a whole refuses to be bound by such interfaces,
sometimes even during a stable kernel series.  

So someone who releases only binary modules will likely be in a world of
hurt.   And that's considered a feature.  Certainly no one is going to
go out of their way to make life easier for binary module authors.

						- Ted
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Alan Cox]

> Actually, he's been quite specific.  It's ok to have binary modules as
> long as they conform to the interface defined in /proc/ksyms.  

What is completely unclear is if he has the authority to say that given that
there is code from other people including the FSF merged into the tree.

I've taken to telling folks who ask about binary modules to talk to their legal
department. The whole question is simply to complicated for anyone else to
work on.

Alan

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Jeff Garzik]

Alan Cox wrote:
> 
> > Actually, he's been quite specific.  It's ok to have binary modules as
> > long as they conform to the interface defined in /proc/ksyms.
> 
> What is completely unclear is if he has the authority to say that given that
> there is code from other people including the FSF merged into the tree.

I've always wondered if we shouldn't encourage people to assign the
copyright for their code to Linux International[1], or some other truly
neutral organization like the FSF[2].  gcc and related projects assign
their copyrights to the FSF to avoid messes like this...

At this point we couldn't force people to change their copyright, but we
can encourage, if LI or another entity exists to which copyrights can be
assigned.

	Jeff


[1] After talking to LI lawyers and getting it all set up, etc.
[2] In this particular case, due to the modified kernel GPL, FSF is
probably a bad target for copyright assignments.

-- 
Jeff Garzik             |
Building 1024           | Would you like a Twinkie?
MandrakeSoft            |
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Theodore Y. Ts'o]

   Date: Thu, 9 Nov 2000 14:26:33 +0000 (GMT)
   From: Alan Cox <alan@lxorguk.ukuu.org.uk>

   > Actually, he's been quite specific.  It's ok to have binary modules as
   > long as they conform to the interface defined in /proc/ksyms.  

   What is completely unclear is if he has the authority to say that given that
   there is code from other people including the FSF merged into the
   tree.

He said it long enough ago that presumably people who merged code in
would have accepted it as an implicit agreement, if it had been
documented in the COPYING file.  Unfortuantely, it wasn't so documented,
so I agree it's unclear.

   I've taken to telling folks who ask about binary modules to talk to
   their legal department. The whole question is simply to complicated
   for anyone else to work on.

... and at that point we run intothe oft-debated (but never tested in a
court of law) question of into the whether or not Copyright can infect
across a link, especially since the link is done by the end-user.  (Yes,
there are some questions about inline functions in the header files.)

The intent of what the FSF would like the case to be is clear, but it's
not clear at all that Copyright law works that way; intent doesn't
matter if the laws don't give you the right to sue on those grounds.
See a lawyer and get official legal advice indeed....


						- Ted

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Alan Cox]

> reason to hamstring their efforts because of the possibility of binary
> modules. The GPL allows that, right? So any developer of binary-only

Its not clear the GPL does allow it. 

> extensions using the GKHI would not be breaking the license agreement, I
> don't think. There's lots of binary modules right now -- VMWare, Aureal
> sound card drivers, etc.

All of which just cause large numbers of bugs to go in the bitbucket because
nobody can tell whose the problem is.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Jesse Pollard]

Larry McVoy <lm@bitmover.com>:
> On Thu, Nov 09, 2000 at 08:44:11AM +0100, Christoph Rohland wrote:
> > Hi Michael,
> > 
> > On Wed, 08 Nov 2000, Michael Rothwell wrote:
> > > Sounds great; unfortunately, the core group has spoken out against a
> > > modular kernel.
> > > 
> > > Perhaps IBM should get together with SGI, HP and other interested
> > > parties and start an Advanced Linux Kernel Project. Then they can
> > > run off and make their scalable, modular, enterprise kernel and the
> > > Linus Version can always merge back in features from it.
> > 
> > *Are you crazy?* =:-0 
> > 
> > Proposing proprietary kernel extensions to establish an enterprise
> > kernel? No thanks!
> 
> Actually, I think this idea is a good one.  I'm a big opponent of all the
> big iron feature bloat getting into the kernel, and if SGI et al want to
> go off and do their own thing, that's fine with me.  As long as Linus 
> continues in his current role, I doubt much of anything that the big iron
> boys do will really make it back into the generic kernel.  Linus is really
> smart about that stuff, are least it seems so to me; he seems to be well
> aware that 99.9999% of the hardware in the world isn't big iron and never
> will be, so something approximating 99% of the effort should be going towards
> the common platforms, not the uncommon ones.

Not strictly true... Many of the capabilities in Linux now came from what
was "big iron" 10 years ago. At the rate that CPU speeds/multi-CPU systems
are becoming available, the current "big iron" will be on your desk, sharing
data among many other systems to solve even bigger problems.

I happen to be a proponent of big iron features...  Just let ME choose which
ones I need, and which I don't.

I already need MLS and VPN support with encryption (IPSec + more :). When my
SGI workstation gets replaced (by a Linux PC) I'll want to be able to
administer (securely) the security structures supported by Cray UNICOS,
Trusted Solaris and Trusted IRIX.

Currently, I have to downgrade the security of the Cray systems just to
administer it (not good, but currently acceptable by the Powers That Be).

I want better. Better security, better authentication, better identification,
secure communication, and auditability. All of this is currently part of "big
iron", but is needed in the commercial world. IPSec is beginning to be called
for in business. MLS is already used there, and I think its use will only
expand. Anything that makes an improvement in this area is needed.

I believe that having the hooks might make it easier to implement higher
levels of security by option. It will certainly make it easer to test
new features. Include a module that attaches to hooks already present --
Other users wouldn't need the module. So let it be an option.

Linux already runs on big iron. Why not support it.

<joke>Anything that can put pressure on M$ ...</joke>

-------------------------------------------------------------------------
Jesse I Pollard, II
Email: pollard@navo.hpc.mil

Any opinions expressed are solely my own.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[richardj_moore]

> The problem with the hooks et.al. is very simple - they promote every
> bloody implementation detail to exposed API. ....

Surely not, having the kernel source does that. The alternative to the hook
is embed a patch in the kernel source.  What proveds greater exposure to
internals: hooks of source?


Richard Moore -  RAS Project Lead - Linux Technology Centre (PISC).

http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd,  MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Alexander Viro]

On Fri, 10 Nov 2000 richardj_moore@uk.ibm.com wrote:

> 
> 
> 
> > The problem with the hooks et.al. is very simple - they promote every
> > bloody implementation detail to exposed API. ....
> 
> Surely not, having the kernel source does that. The alternative to the hook
> is embed a patch in the kernel source.  What proveds greater exposure to
> internals: hooks of source?

Sorry. You don't "embed" the patch. You either get it accepted or not.
Or you fork the tree and then it's officially None Of My Problems(tm).
If your private patch breaks because of the kernel changes - too fscking
bad, it's still None Of My Problems(tm). With hooks you have a published
API.

Alternative to hook is not a random patch. It's finding a way to use public
APIs (possibly at the cost of redesigning your code) or finding good arguments
for changing said APIs (ditto). And they'ld better be really good. "I don't
know how to do that without rethinking my code" doesn't cut it. Deal. There
is _no_ warranty that anything other than public APIs will not change at any
random moment. Period. You play with layering violations - you _will_ shoot
yourself in foot. It's not "if", it's "when".

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Michael Rothwell]

Alexander "see figure 1" Viro wrote:

> Sorry. You don't "embed" the patch. You either get it accepted or not.
> Or you fork the tree and then it's officially None Of My Problems(tm).

Sounds like a good idea. 

-M
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Alexander Viro]

On Fri, 10 Nov 2000, Michael Rothwell wrote:

> > Sorry. You don't "embed" the patch. You either get it accepted or not.
> > Or you fork the tree and then it's officially None Of My Problems(tm).
> 
> Sounds like a good idea. 

It's not a good idea, it's an obvious fact. Oh, you mean forking the tree?
Sure, go ahead - license explicitly allows that. You have to maintain the
thing afterwards, but that's kinda obvious.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[David Lang]

how is that any different then a module? modules that are not included
with the kernel source are not guarenteed to work with any other kernel
version (including during the stable kernel series)

David Lang

On Fri, 10 Nov 2000, Alexander Viro wrote:

> On Fri, 10 Nov 2000 richardj_moore@uk.ibm.com wrote:
> 
> > > The problem with the hooks et.al. is very simple - they promote every
> > > bloody implementation detail to exposed API. ....
> > 
> > Surely not, having the kernel source does that. The alternative to the hook
> > is embed a patch in the kernel source.  What proveds greater exposure to
> > internals: hooks of source?
> 
> Sorry. You don't "embed" the patch. You either get it accepted or not.
> Or you fork the tree and then it's officially None Of My Problems(tm).
> If your private patch breaks because of the kernel changes - too fscking
> bad, it's still None Of My Problems(tm). With hooks you have a published
> API.
> 
> Alternative to hook is not a random patch. It's finding a way to use public
> APIs (possibly at the cost of redesigning your code) or finding good arguments
> for changing said APIs (ditto). And they'ld better be really good. "I don't
> know how to do that without rethinking my code" doesn't cut it. Deal. There
> is _no_ warranty that anything other than public APIs will not change at any
> random moment. Period. You play with layering violations - you _will_ shoot
> yourself in foot. It's not "if", it's "when".
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> Please read the FAQ at http://www.tux.org/lkml/
> 
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[richardj_moore]

>>  Why? I think the IBM GKHI code would be of tremendous value. It would
....
>
> And we already refuse to support those kernels - your point being?
>
> Making this "commonplace" is a nightmare. Go away with that.


How is so?


Richard Moore -  RAS Project Lead - Linux Technology Centre (PISC).

http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd,  MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[richardj_moore]

> Yes, and that's why I am opposing here: Technically you are right, but
> proposing that enterprise Linux should go this way is inviting binary
> only modules due to the lax handling of modules.

Not so sure it does. If a kernel module wants to make use of GKHI then it
will have to

1) include a GKHI header file or copy some of the code in it,
2) Update kernel source in a minimal way to add the callbacks

Wouldn't 1) under GPL terms force the kernel module to be  GPL?



Richard Moore -  RAS Project Lead - Linux Technology Centre (PISC).

http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd,  MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[richardj_moore]

>> extensions using the GKHI would not be breaking the license agreement, I
>> don't think. There's lots of binary modules right now -- VMWare, Aureal
> sound card drivers, etc.
>
>All of which just cause large numbers of bugs to go in the bitbucket
because
>nobody can tell whose the problem is.


I don't understand your point - are you saying that the existence of kernel
modules causes makes problems more difficult to solve. Why would that be?


Richard Moore -  RAS Project Lead - Linux Technology Centre (PISC).

http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd,  MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[richardj_moore]

> That being said, the real problem with the GKHI is that as Al said, it
> does expose internal kernel interfaces --- and the Linux kernel
> development community as a whole refuses to be bound by such interfaces,
> sometimes even during a stable kernel series.

I'm not sure that GKHI exposes any more interfaces than embedding a patch
directly into the kernel would.

It has the potential to to make patches easier to re-work for different
kernel versions, and to enable development maintence and fixing of the
patch to be done independently of a kernel build. And it also has the
potential of helping with co-existence. If for example the RAS community
could agree on a number of hooks (I'm thinking here of crash dump, trace,
dprobes and maybe KDB as well) then you'd probably find a good may on them
using then same hooks. The modifications to the kernel would be minimal and
the user would be left an easy means of installing a co-existing subset of
the offerings supported by hooks.

An example: DProbes is down to three hooks - that's three lines of code in
the kernel + three lines in ksyms.c

Patching DProbes onto any custom kernel is a doddle.


Richard Moore -  RAS Project Lead - Linux Technology Centre (PISC).

http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd,  MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Theodore Y. Ts'o]

   From: richardj_moore@uk.ibm.com
   Date: 	Fri, 10 Nov 2000 11:41:09 +0000

   It has the potential to to make patches easier to re-work for different
   kernel versions, and to enable development maintence and fixing of the
   patch to be done independently of a kernel build. And it also has the
   potential of helping with co-existence. If for example the RAS community
   could agree on a number of hooks (I'm thinking here of crash dump, trace,
   dprobes and maybe KDB as well) then you'd probably find a good may on them
   using then same hooks. The modifications to the kernel would be minimal and
   the user would be left an easy means of installing a co-existing subset of
   the offerings supported by hooks.

   An example: DProbes is down to three hooks - that's three lines of code in
   the kernel + three lines in ksyms.c

   Patching DProbes onto any custom kernel is a doddle.

Right.  So what you're saying is that GKHI is adding complexity to the
kernel to make it easier for peopel to put in non-standard patches which
exposes non-standard interfaces which will lead to kernels not supported
by the Linux Kernel Development Community.  Right?

Alternatively, you can argue for specific hooks, and try to see if you
can convince the Kernel Developers (and Linus Torvalds, in
particular) that such hooks are good public interfaces, and that adding
such interfaces are a Good Thing.  If such hooks are agreed to, then the
GKHI might be a good wya of implementing these hooks.

But if there are no standard hooks in the mainline kernel, it's going to
be hard to pursuade people that adding the GKHI would be a good thing.
So for the purposes of getting GKHI into the kernel, argueing for GKHI
in the abstract is putting the card before the horse.  What I would
recommend is showing how certain hooks are good things to have in the
mainline kernel, and to try to pursuade people of that question first.
Only then try to argue for GKHI.

						- Ted

P.S.  There are some such RAS features which I wouldn't be surprised
there being interest in having integrated into the kernel directly
post-2.4, with no need to put in "kernel hooks" for that particular
feature.  A good example of that would be kernel crash dumps.  For all
Linux houses which are doing support of customers remotely, being able
to get a crash dump so that developers can investigate a problem
remotely instead of having to fly a developer out to the customer site
is invaluable.  In fact, it might be considerd more valuable than the
kernel debugger....
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Christoph Rohland]

Hi Theodore,

On Fri, 10 Nov 2000, Theodore Y. Ts'o wrote:
> P.S.  There are some such RAS features which I wouldn't be surprised
> there being interest in having integrated into the kernel directly
> post-2.4, with no need to put in "kernel hooks" for that particular
> feature.  A good example of that would be kernel crash dumps.  For
> all Linux houses which are doing support of customers remotely,
> being able to get a crash dump so that developers can investigate a
> problem remotely instead of having to fly a developer out to the
> customer site is invaluable.  In fact, it might be considerd more
> valuable than the kernel debugger....

*Yes* :-)

Greetings
		Christoph

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Matt D. Robinson]

Christoph Rohland wrote:
> 
> Hi Theodore,
> 
> On Fri, 10 Nov 2000, Theodore Y. Ts'o wrote:
> > P.S.  There are some such RAS features which I wouldn't be surprised
> > there being interest in having integrated into the kernel directly
> > post-2.4, with no need to put in "kernel hooks" for that particular
> > feature.  A good example of that would be kernel crash dumps.  For
> > all Linux houses which are doing support of customers remotely,
> > being able to get a crash dump so that developers can investigate a
> > problem remotely instead of having to fly a developer out to the
> > customer site is invaluable.  In fact, it might be considerd more
> > valuable than the kernel debugger....
> 
> *Yes* :-)

As soon as I finish writing raw write disk routines (not using kiobufs),
we can _maybe_ get LKCD accepted one of these days, especially now that we
don't have to build 'lcrash' against a kernel revision.  I'm in the
middle of putting together raw IDE functions now -- see LKCD mailing
list for details if you're curious.

IMHO, GKHI is a good thing -- it would be great to see this used for
ASSERT() cases (something you can turn on by 'insmod assert.o', which
would then trigger assert conditionals throughout the kernel ...) I
realize it would mean some bloat, and I doubt Linus would accept it,
but it's a nifty concept for enterprise Linux servers (especially
those that want quick answers to system crashes).

--Matt

> Greetings
>                 Christoph
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Theodore Y. Ts'o]

   Date: Fri, 10 Nov 2000 10:36:31 -0800
   From: "Matt D. Robinson" <yakker@alacritech.com>

   As soon as I finish writing raw write disk routines (not using kiobufs),
   we can _maybe_ get LKCD accepted one of these days, especially now that we
   don't have to build 'lcrash' against a kernel revision.  I'm in the
   middle of putting together raw IDE functions now -- see LKCD mailing
   list for details if you're curious.

Great!  Are you thinking about putting the crash dumper and the raw
write disk routines in a separate text section, so they can be located
in pages which are write-protected from accidental modification in case
some kernel code goes wild?  (Who me?  Paranoid?  :-)

						- Ted
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Matt D. Robinson]

"Theodore Y. Ts'o" wrote:
> 
>    Date: Fri, 10 Nov 2000 10:36:31 -0800
>    From: "Matt D. Robinson" <yakker@alacritech.com>
> 
>    As soon as I finish writing raw write disk routines (not using kiobufs),
>    we can _maybe_ get LKCD accepted one of these days, especially now that we
>    don't have to build 'lcrash' against a kernel revision.  I'm in the
>    middle of putting together raw IDE functions now -- see LKCD mailing
>    list for details if you're curious.
> 
> Great!  Are you thinking about putting the crash dumper and the raw
> write disk routines in a separate text section, so they can be located
> in pages which are write-protected from accidental modification in case
> some kernel code goes wild?  (Who me?  Paranoid?  :-)
> 
>                                                 - Ted

We're planning to isolate the write functions as much as possible.
In the past, we've been bitten by this whole concept of Linux "raw I/O".
When I was at SGI, we were able to write to a block device directly
through low-level driver functions that weren't inhibited by any
locking, and that was after shutting down all processors and any
other outstanding interrupts.  For Linux, I had given up and stuck
with the raw I/O interpretation of kiobufs, which is just flat out
wrong to do for dumping purposes.  Secondly, as Linus said to me a
few weeks ago, he doesn't trust the current disk drivers to be able
to reliably dump when a crash occurs.  Don't even ask me to go into
all the reasons kiobufs are wrong for crash dumping.  Just read
the code -- it'll be obvious enough.

So I guess after a few months/years, we're finally at a point where
we're saying, "Okay, forget all this, let's do this the right way,
so we don't have these problems anymore."  We're removing lcrash from
the kernel, putting it into its own RPM, and adding patches to the
kernel for LKCD that build in crash dump functionality and make a new
"Kernsyms" file so that we can dynamically read the symbol table of
major parts of the kernel and give you memory dumps, stack traces,
and even dump out entire structures dynamically.  Then we'll have
the right crash dump mechanism for everyone.

It's time to get RAS moving for Linux.  GKHI and LKCD are the first
steps to get us there (IMHO).

--Matt
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Keith Owens]

On Fri, 10 Nov 2000 19:29:26 -0800, 
"Matt D. Robinson" <yakker@alacritech.com> wrote:
>We're removing lcrash from
>the kernel, putting it into its own RPM, and adding patches to the
>kernel for LKCD that build in crash dump functionality and make a new
>"Kernsyms" file so that we can dynamically read the symbol table of
>major parts of the kernel and give you memory dumps, stack traces,
>and even dump out entire structures dynamically.

kallsyms goes a long way towards solving the symbol table problem for
debugging.  It really only has three deficiencies, it does not detail
structure fields, it does not handle automatic variables and it does
not have source line numbers.  All of those need the sort of detail
provided by gcc -g, but the amount of data that generates is
prohibitively large, 40+ megabytes is a bit much to load into kernel
space.  I reluctantly decided that printing global addresses and
offsets was the best I could do, given the space constraints.

Instead of inventing your own kernsyms file, take a look at kallsyms.
It handles modules as well as the kernel.  Let me know if you want any
additional data in kallsyms.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[tytso]

   Date: Fri, 10 Nov 2000 19:29:26 -0800
   From: "Matt D. Robinson" <yakker@alacritech.com>

   We're planning to isolate the write functions as much as possible.
   In the past, we've been bitten by this whole concept of Linux "raw I/O".
   When I was at SGI, we were able to write to a block device directly
   through low-level driver functions that weren't inhibited by any
   locking, and that was after shutting down all processors and any
   other outstanding interrupts.  For Linux, I had given up and stuck
   with the raw I/O interpretation of kiobufs, which is just flat out
   wrong to do for dumping purposes.  Secondly, as Linus said to me a
   few weeks ago, he doesn't trust the current disk drivers to be able
   to reliably dump when a crash occurs.  Don't even ask me to go into
   all the reasons kiobufs are wrong for crash dumping.  Just read
   the code -- it'll be obvious enough.

Oh, yeah, I could have told you that from the beginning.  kiobufs were
never intended to be crash-dump friendly.  :-)   My preference would be
that each block device that was going to be doing crash dumping would
use a special busy-looping driver that's guaranteed never to fail.
(Sort of like how the serial console driver is done; it's separate from
the rest of the driver, and does not depend on interrupts working.)
Hence my comment about putting that separate bit of code in a page which
is write-protected and segregated from everything else....

							- Ted
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Daniel Phillips]

tytso@mit.edu wrote:
> 
>    Date: Fri, 10 Nov 2000 19:29:26 -0800
>    From: "Matt D. Robinson" <yakker@alacritech.com>
> 
>    We're planning to isolate the write functions as much as possible.
>    In the past, we've been bitten by this whole concept of Linux "raw I/O".
>    When I was at SGI, we were able to write to a block device directly
>    through low-level driver functions that weren't inhibited by any
>    locking, and that was after shutting down all processors and any
>    other outstanding interrupts.  For Linux, I had given up and stuck
>    with the raw I/O interpretation of kiobufs, which is just flat out
>    wrong to do for dumping purposes.  Secondly, as Linus said to me a
>    few weeks ago, he doesn't trust the current disk drivers to be able
>    to reliably dump when a crash occurs.  Don't even ask me to go into
>    all the reasons kiobufs are wrong for crash dumping.  Just read
>    the code -- it'll be obvious enough.
> 
> Oh, yeah, I could have told you that from the beginning.  kiobufs were
> never intended to be crash-dump friendly.  :-)   My preference would be
> that each block device that was going to be doing crash dumping would
> use a special busy-looping driver that's guaranteed never to fail.
> (Sort of like how the serial console driver is done; it's separate from
> the rest of the driver, and does not depend on interrupts working.)
> Hence my comment about putting that separate bit of code in a page which
> is write-protected and segregated from everything else....

This is also needed for single-machine source level kernel debugging.

--
Daniel
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Lars Marowsky-Bree]

On 2000-11-10T19:12:29,
   "Theodore Y. Ts'o" <tytso@MIT.EDU> said:

> Great!  Are you thinking about putting the crash dumper and the raw
> write disk routines in a separate text section, so they can be located
> in pages which are write-protected from accidental modification in case
> some kernel code goes wild?  (Who me?  Paranoid?  :-)

I would also suggest to have a little checksum over the relevant pages, to
verify that the code is still correct and we are not going to crashdump all
over our valuable data...

And I am still very fond of the idea of crash dumping to a network server ;-)

Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>
    Development HA

-- 
Perfection is our goal, excellence will be tolerated. -- J. Yahl

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Michael Rothwell]

Lars Marowsky-Bree wrote:

> And I am still very fond of the idea of crash dumping to a network server ;-)

I second that. Serial can be slow, and has that pesky cable-length
limit...
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Andi Kleen]

On Fri, Nov 10, 2000 at 11:24:28AM -0500, Theodore Y. Ts'o wrote:
> Right.  So what you're saying is that GKHI is adding complexity to the
> kernel to make it easier for peopel to put in non-standard patches which
> exposes non-standard interfaces which will lead to kernels not supported
> by the Linux Kernel Development Community.  Right?

My understanding is that GKHI does not change the kernel at all, except
for the three hooks needed for dprobes. All GKHI hooks are implemented 
as dynamic probes, which are just like debugger breakpoints. 
A dynamic probes breakpoint does not require any source
changes, but you have to check the assembly to find the right point for
them (at least in the current version, I don't know if IBM is planning 
to support source level dprobes using the debugging information) 

IMHO GKHI does not make mainteance of additional modules any easier, because
you always have to recheck the assembly if the dynamic probe still fits
(which may in some cases even be more work than reporting source patches,
it is also harder when you want to cover multiple architectures) 

It will just help some people who have a unrational aversion against kernel
recompiles and believe in vendor blessed binaries.


-Andi

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[richardj_moore]

> Right.  So what you're saying is that GKHI is adding complexity to the
> kernel to make it easier for peopel to put in non-standard patches which
> exposes non-standard interfaces which will lead to kernels not supported
> by the Linux Kernel Development Community.  Right?

I don't think I mentioned complexity - in what was do you mean adding
complexity? If you mean addition funciton then yes. If you mean kernel
source modificaiton then no. If you mean the mechanism then it's nothing
special - just what a loader does when it fixes up an external reference.

> But if there are no standard hooks in the mainline kernel, it's going to
> be hard to pursuade people that adding the GKHI would be a good thing.
> So for the purposes of getting GKHI into the kernel, argueing for GKHI
> in the abstract is putting the card before the horse.  What I would
> recommend is showing how certain hooks are good things to have in the
> mainline kernel, and to try to pursuade people of that question first.
> Only then try to argue for GKHI.

Quite agree, there are no standard hooks, no hooks at all in fact. I'm
neither seeking to get this accepted or not accepted into the kernel. What
it does do is give me an easier time both as a developer and an installer
when I want to include some rarefied code additions. GKHI was developed
primarily for DProbes.

> P.S.  There are some such RAS features which I wouldn't be surprised
> there being interest in having integrated into the kernel directly
> post-2.4,

Great!

Richard Moore -  RAS Project Lead - Linux Technology Centre (PISC).

http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd,  MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[richardj_moore]

No, misunderstood.

GKHI is not implemented using dynamic probes.  GKHI places in the kernel
calls to APIs in the DProbes code. Since we'ed rather have Dprobes out of
the kernel then essentially it acts as a loader after the fact, i.e. it
fixes up the DProbes API calls when the DProbe module loads. Compare this
with the usual loading process where the fix-ups are done in the module
being loaded. Now, you might want to ask me why I want DProbes as a module?
They again you might not. Either way is fine by me ;-)


Richard Moore -  RAS Project Lead - Linux Technology Centre (PISC).

http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd,  MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK


Andi Kleen <ak@suse.de> on 10/11/2000 16:54:05

Please respond to Andi Kleen <ak@suse.de>

To:   "Theodore Y. Ts'o" <tytso@MIT.EDU>
cc:   Richard J Moore/UK/IBM@IBMGB, Paul Jakma <paulj@itg.ie>, Michael
      Rothwell <rothwell@holly-springs.nc.us>, Christoph Rohland
      <cr@sap.com>, linux-kernel@vger.kernel.org
Subject:  Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)




On Fri, Nov 10, 2000 at 11:24:28AM -0500, Theodore Y. Ts'o wrote:
> Right.  So what you're saying is that GKHI is adding complexity to the
> kernel to make it easier for peopel to put in non-standard patches which
> exposes non-standard interfaces which will lead to kernels not supported
> by the Linux Kernel Development Community.  Right?

My understanding is that GKHI does not change the kernel at all, except
for the three hooks needed for dprobes. All GKHI hooks are implemented
as dynamic probes, which are just like debugger breakpoints.
A dynamic probes breakpoint does not require any source
changes, but you have to check the assembly to find the right point for
them (at least in the current version, I don't know if IBM is planning
to support source level dprobes using the debugging information)

IMHO GKHI does not make mainteance of additional modules any easier,
because
you always have to recheck the assembly if the dynamic probe still fits
(which may in some cases even be more work than reporting source patches,
it is also harder when you want to cover multiple architectures)

It will just help some people who have a unrational aversion against kernel
recompiles and believe in vendor blessed binaries.


-Andi




-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[richardj_moore]

Matti,


>    Please educate me, what does "our RAS offerings" mean here ?
>    (I didn't find "RAS" at your signature-URL site, but I didn't
>     poke around very much..)

RAS = Reliabilty, Availability & Serviceability = those things that are are
not mainline to an OS but add the qualities named in the acronym. That
includes self-healing, recoverability, diagnosis etc..
My specialism is in probelm diagnosis. When I say RAS I generally mean
debugging/diagnosis aids, but I also mean it not from a developement
standpoint but from a support standpoint. Which depending on how a system
is deployed can be very different things.

>    I do know that when IBM suits speak with phrases like that,
>    they are selling me something which costs $$$.

No always. All this stuff (Linux RAS) is free and given away under GPL. And
I'm not waring a suit either ;-) RAS in general is not sexy, it's difficult
to sell. So it's pad for from after sales service and other indirect means.

>    Which definitely gives proprietary, binary only, hook image...
>    But GKHI, and DProbes are neither. Thus I am confused, but can
>    understand the furor...

Well I sort of can and can't as well. Here's a couple of circumstances
where I'd find GKHI useful:

I'm developing DProbes, I need the SGI KDB, a complex patch, as a debugging
aid. I also want to keep up with various kernel version. I've got limited
time so would like to spend it on DProbes on not re-working patches. So, I
know that DProbes only needs to get control in three places in kernel
processing:
1) Trap 1 handler
2) Trap 3 handler
3) Pagefault handler.

I reason that if I had a call inserted into the kernel source at these
points to respective entry points in my DProbes code I'd not have to spend
much time integrating SGI's KDB with DProbes.
The I realise if I leave the calls nop'd and dynamically patch them in
later I can build the kernel once and re-build DProbes (now a module) many
times over - and sometimes without re-booting.

So I create GKHI to mess around with NOPs converitng them into calls.

Second scenario:
I have a customer running Linux for a business purpose. They are not
developers and have no programming skill. Every now and again their system
crashes and they have to reboot. And down time costs them in real terms.

So they say to IBM, you supplied this ... system, you fix it. OK we say,
we'll need a dump. We'll send you a kernel with SGI's crash dump built in.
On no you won't they say, you're not sending us any more dodgy code until
you've fixed this problem. Anyway the server is in a secure remote branch
office. There's no technoical support on site and we cannot possibly have
developers messing with that system. Now suppose SGI or IBM have converted
SGI Kernel Crash Dump to a module and we supplly the system customised with
a few GKHI hooks in place. Then we say issue the following command: insmod
lkcd.o

We get a dump and discover that some cheesehead had overlaid a spinlock
causing re-entrancy and a crash. OK we say we know what happened by not who
did it. So, we need to trace all storage alterations to the spin-lock.
There are only a few valid user's of that spin-lock, which if we had a
trace, we could eliminate. By now DProbes and Linux Trace Toolkit are
working well together, and providing a dynamic trace capability. Also
DProbes is now offering the capability of probes on storage modifications.
So we say to the customer please issue three commands: insmod lkcd; insmod
dprobes; insmod ltt;

The system crashes, we get the trace. We duly find the address from which
the spin-lock was overwriten. We look in the dump and find it's a routine
in a device driver that's been passed invalid data, but actually, not
passed, but placed on a work queue. And furthermore the invalid data has a
particular look to it.

We explain to the customer that we now need just one more pice of
information. So finally we place a dprobe on the enqueuing routine, looking
at data enqueued until the invalid pattern occurs and make the probe
trigger another dump.

And finally we have it. The enqueuing routine was another driver .....

This scenario is not untypical of the sort of problem I earnt my living
solving for the past n years.

Now we could have supplied a system with Crash Dump, Dprobes, LTT, KDB, a
dozen other specialised RAS tools. The kernel would have 50% bigger, cost
us considerable time and effort whenever kernel maintenance was applied -
with obvious consequences. And in the end 99.9% of the time we don't need
these facilities, coz after all Linux is a pretty stable platform. So why
not allow them to be brought in dynamically.



Richard Moore -  RAS Project Lead - Linux Technology Centre (PISC).

http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd,  MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[richardj_moore]

Andi Kleen wrote:
> It will just help some people who have a unrational aversion against
kernel
>recompiles and believe in vendor blessed binaries.


An interesting remark Andi, especially in the light of your note to me
regarding your use of DProbes - i.e. you'd rather use DProbes to dump out
some info from the kernel than recompile it with printks.

I don't have an aversion to recompiling the kernel - it's great fun - I
love watching all the meeages go by, waiting with bated breath for a
compile error, which never seems to happen. Just like watching the National
Lottery, waiting for your own numbers to come up.

To be a little more serious, it's not recompilation that's a problem, its
re-working a set of (non-standard) patches together. I'm not that excited
by that - I'd rather develop new code than rework old. Anyway for a couple
of  example scenarios see the response I made to Michael Rothwell.  And by
the way, I absolutely agree with your approach to kernel problem solving -
but wouldn't it be a help if you didn't have to put a large or even
moderate effort into working the DProbes patch into some hot-off-the-press
version of the kernel?

Richard


Richard Moore -  RAS Project Lead - Linux Technology Centre (PISC).

http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd,  MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Andi Kleen]

On Sun, Nov 12, 2000 at 11:27:26PM +0000, richardj_moore@uk.ibm.com wrote:
> 
> 
> 
> Andi Kleen wrote:
> > It will just help some people who have a unrational aversion against
> kernel
> >recompiles and believe in vendor blessed binaries.
> 
> 
> An interesting remark Andi, especially in the light of your note to me
> regarding your use of DProbes - i.e. you'd rather use DProbes to dump out
> some info from the kernel than recompile it with printks.

When I wrote it I was still misunderstanding GKHI's nature (I was assuming
that it worked on top of dprobes, not under it -- I should have read the 
source before commenting, my bad)

I think using dprobes for collecting information is ok, but when you want 
to do actual actions with it (not only using it as a debugger) IMHO it 
is better to patch and recompile the kernel. 

> 
> I don't have an aversion to recompiling the kernel - it's great fun - I
> love watching all the meeages go by, waiting with bated breath for a
> compile error, which never seems to happen. Just like watching the National
> Lottery, waiting for your own numbers to come up.
> 
> To be a little more serious, it's not recompilation that's a problem, its
> re-working a set of (non-standard) patches together. I'm not that excited
> by that - I'd rather develop new code than rework old. Anyway for a couple
> of  example scenarios see the response I made to Michael Rothwell.  And by
> the way, I absolutely agree with your approach to kernel problem solving -
> but wouldn't it be a help if you didn't have to put a large or even
> moderate effort into working the DProbes patch into some hot-off-the-press
> version of the kernel?

As far as I can see GKHI is overkill for dprobes alone, the existing
notifier lists would be sufficient because dprobes does not hook into any 
performance critical paths. 


-Andi
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Andrea Arcangeli]

On Mon, Nov 13, 2000 at 11:38:23AM +0100, Andi Kleen wrote:
> notifier lists would be sufficient because dprobes does not hook into any 
> performance critical paths. 

Current dprobes patch adds branches in the the main page fault handler,
device_not_available exception at least. Those are _very_ hot paths.

I had a look at gkhi and infact my guess is that the main reason of
implementing gkhi in first place is probably because they had to remove the
branches from the fast paths when dprobes is inactive by using self modifying
code to only have some additional nop in the exception fast paths during
production, and while doing that they probably choosed to generalize the
lightweight self modifying hook interface instead of doing a magic inside the
dprobes patch.

But (unless I'm missing something in the code :) gkhi has absolutely _nothing_
new with respect to binary modules except being able to introduce less costly
hooks in term of performance _only_when_ the hooks are unregistered (= when
machine is in production). It's only a matter of performance as far I can see.
To introduce a new hook using gkhi it's necessary to change the kernel sources,
recompile the kernel just like for every other module registration hook that we
just have in 2.4.0-test11-pre2. The modules using gkhi interface needs the hook
details (were the self modifying code is placed) exported via /proc/ksyms
infact, just like for any other regular hook registered via the init_module
without using self modifying code.

So binary only modules using gkhi _can't_ have _any_ "binary level" advantage
compared to all other regular binary only modules for 2.2.x and 2.4.x that
doesn't use gkhi.

And even if they did the other way around - so having gkhi on top of dprobes -
they would not only been dependent on symbols, but they would depend on almost
everything including a recompile of the kernel that doesn't affect the symbols
and they would be still add branches in the exception handlers as current
dprobe patch is doing. Not to tell that the dprobes hooks are very inefficient
and unsuitable for productions if they're recalled at high frequency in fast
paths since there are all generating CPU exceptions that have the same latency
of an enter/exit kernel every time an hook triggers plus there's the
interpreter that must execute the hook.

I think gkhi is good idea to be able to implement lightweight hooks that are
unregistered during production as it avoid the cost of dereferencing pointers
in a linked list to know if we have any hook registered or not.  dprobes is
perfect user of the lightweight gkhi interface. However they're useful __only__
when the hook is unregistred all the time during production.  For example
replacing the binfmt chain with gkhi would decrease performance because
gkhi force the hook handling to happen out of line. I don't think there's any
hook in a hot path and unregistered all the time in the current kernels so I
think only point of merging gkhi would be only to merge a more efficient
implementation of dprobes.

I think gkhi should be renamed to something like "Fast Unregistered Kernel
Hook Interface" to avoid confusion and wrong usage of it that would otherwise
leads to lower performance.

I've a few comments about the implementation:

__asm__ volatile (".global GKHook"h"; 
		.global GKHook"h"_ret; 
		.global GKHook"h"_bp; 
		GKHook"h": jmp GKHook"h"_bp; /* nop; to activate */  
			   ^^^^^^^^^^^^^^^^
			push $0; 
			nop;nop;nop;nop;nop; /* jmp GKHook"h"_dsp when active*/
	 	GKHook"h"_ret: add $4,%%esp;   
		GKHook"h"_bp:;":::"%eax")
				   ^^^^

1)

It would be better to have only 5 nop instructions (instead of more
instructions and a jmp over them) and patch them at runtime as soon as somebody
registers the hook to generate a `call hook_handler_1'. This will also save
icache.

For the atomicity of the operation in SMP (of the update of the 5 nops to call
the dispatcher) we should probably have a fixup entry in place for the GKHook
address, so that we can fixup the 5th byte after lefting an invalid opcode at
the first byte.

2)

eax needs to be saved internally to the hook. We must only add the 5 nop
penality in the unregistered case and primarly on x86 with so few regs we
really want to have the fast path not having to save/restore eax on the stack
across every nop gkhi hook.

3)

gkhi apparently doesn't yet know the word "SMP" 8).

4)

gkh_iflush should be done with flush_icache_range that is infact implemented
wrong for IA32 and it should be implemented as regs trashing cpuid (the fact
it's wrongly implemented means that in theory modules can break in IA32
on 2.4.x 2.2.x even on UP).

5)

Current dprobes v1.1.1 against 2.2.16 cames with a syscall collision:
sys_dprobes collides with ugetrlimit (not implemented in 2.2.x). That's fine
for internal use and to show the code, but make _sure_ not to ship any binary
to anybody implementing ugetrlimit as sys_dprobes 8).

Richard please ask Linus to reserve a syscall for dprobes. I recommend to
allocate the syscall out of the way (I mean using syscall 255 or even better
enlarging the syscall table from 256 to 512 and using number 511) so we make
sure not to waste precious dcachelines for debugging stuff.

BTW, for things like lkcd there's no reason to use gkhi to make it completly
modular since lkcd gets recalled in a completly slow path.

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[richardj_moore]

Alexander Viro wrote:
> It's not a good idea, it's an obvious fact. Oh, you mean forking the
tree?

Again I find your terminology at odds with mine; what do you mean by
forking the tree? I get the impression that it's a very restrictive notion
where any functional ehancement applied as a patch on top of a standard
distribution kernel is considered by you as forking? Is that so? (And BTW
by patch I mean input to the patch command.)


Richard Moore -  RAS Project Lead - Linux Technology Centre (PISC).

http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd,  MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[richardj_moore]

Andi Kleen wrote:
>I think using dprobes for collecting information is ok, but when you want
>to do actual actions with it (not only using it as a debugger) IMHO it
>is better to patch and recompile the kernel.

I absolutely agree. The only time I ever used this capability was to modify
a proprietary binary, for which I did not have the source, so that I could
prove to the owner what needed fixing.

>As far as I can see GKHI is overkill for dprobes alone, the existing
>notifier lists would be sufficient because dprobes does not hook into any
>performance critical paths.

Again, I agree. My intent is that the RAS guys might club together - then
GKHI make much more sense.



Richard Moore -  RAS Project Lead - Linux Technology Centre (PISC).

http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd,  MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[richardj_moore]

Andrea,

I am very greatful for your detailed analysis. I have yet to digest
everything you commented but will get back to you on all points you raise
very soon. Here are my thoughts so far:

> I think gkhi should be renamed to something like "Fast Unregistered
Kernel
> Hook Interface" to avoid confusion and wrong usage of it that would
otherwise
> leads to lower performance.



A fair point.


On point 3):

> 3)
>
> gkhi apparently doesn't yet know the word "SMP" 8).


When I announced GKHI I did state that SMP support was to follow. The
updates are trivial but I didn't wan't to release the code until I had had
a chance to test it.

On point 4)

> 4)
>
> gkh_iflush should be done with flush_icache_range that is infact
implemented
> wrong for IA32 and it should be implemented as regs trashing cpuid (the
fact
> it's wrongly implemented means that in theory modules can break in IA32
> on 2.4.x 2.2.x even on UP).


Are you claiming that flush_icache_range has an error and should implement
the IA32 instruction flush as I did using CPUID? If this is the case has
this error been officially reported?

On point 5)
5)

> Current dprobes v1.1.1 against 2.2.16 cames with a syscall collision:
> sys_dprobes collides with ugetrlimit (not implemented in 2.2.x). That's
fine
> for internal use and to show the code, but make _sure_ not to ship any
binary
> to anybody implementing ugetrlimit as sys_dprobes 8).
>
> Richard please ask Linus to reserve a syscall for dprobes. I recommend to
> allocate the syscall out of the way (I mean using syscall 255 or even
better
> enlarging the syscall table from 256 to 512 and using number 511) so we
make
> sure not to waste precious dcachelines for debugging stuff.

Thanks for this information. Reserving a syscall will become irrelvant when
we release Dprobes as a module using gkhi since we will use ioctl() as the
application interface.

> BTW, for things like lkcd there's no reason to use gkhi to make it
completly
> modular since lkcd gets recalled in a completly slow path.

Well, not necessarily so while lkcd is not get accepted into the standard
kernel source. But also, even when lkcd becomes accepted, using gkhi with
lkcd will allow a crash dump capability to be actived dynamically. That
gives the user more fexibility. Even enterprise customers can sometimes
hedge their bets when it comes to RAS-like features.



Richard Moore -  RAS Project Lead - Linux Technology Centre (PISC).

http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd,  MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[richardj_moore]

Please respond to Andrea Arcangeli <andrea@suse.de>

To:   Richard J Moore/UK/IBM@IBMGB
cc:
Subject:  Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)




On Wed, Nov 15, 2000 at 05:14:57AM +0000, richardj_moore@uk.ibm.com wrote:
>
>
> Andrea,
>
> I am very greatful for your detailed analysis. I have yet to digest
> everything you commented but will get back to you on all points you raise
> very soon. Here are my thoughts so far:

I'm glad you appreciated my comments. I think dprobes gives an higher
level of flexibility for debugging purposes and I'd really like to
include it in the aa kernels until it will be included into the mainstream.

> When I announced GKHI I did state that SMP support was to follow. The

I probably overlooked that part of your announcement, sorry.

> updates are trivial but I didn't wan't to release the code until I had
had
> a chance to test it.

Very promising.

Note that SMP could introduce non trivial issues: the self modifying
changes
should be atomic with respect of the other CPUs executing the self
modifying
code and specs are often not very explicit about side effects of self
modifying
code in SMP, it's not only a matter of implementing the GKHI locks with SMP
locks).

> Are you claiming that flush_icache_range has an error and should
implement
> the IA32 instruction flush as I did using CPUID? If this is the case has

Exactly.

> this error been officially reported?

I hope I did that in my email :). Actually when I fixed the alpha port
some month ago (alpha needs an explicit imb() to flush the speculative
icache and it was really crashing in modules because of the missing
smp_imb) I
also noticed IA32 was buggy, since also IA32 execute out of order and specs
says we must do a cpuid as only way to serialize the istream.  But I didn't
fixed IA32 because nobody ever got bitten by that race because of
timing/implementation reasons, but to be correcet we should do cpuid also
in
flush_icache_range.

Once flush_icache_range is fixed in IA32 you can use it inside GKHI too
(and then you'll get it right on all architectures).

> Thanks for this information. Reserving a syscall will become irrelvant
when
> we release Dprobes as a module using gkhi since we will use ioctl() as
the
> application interface.

Ok (still you need to reserve a blockdevice major minor number with Linus
though).

> Well, not necessarily so while lkcd is not get accepted into the standard
> kernel source. [..]

It won't until it uses a separate driver that doesn't depend on scsi or
ide layer.

Even ignoring the safety problem of scsi layer potentially corrupted by
memory corruption at crash time, the scsi layer doesn't work without being
interrupt driven. It will recurse on the stack badly if somebody ever tries
to
use it polled. Probably similar thing happens with IDE (but none IDE polled
hardware exists so we don't know). I documented all this in the `Linux
Kernel
Debugging' document on my ftp area in ftp.suse.com.

> [.] But also, even when lkcd becomes accepted, using gkhi with
> lkcd will allow a crash dump capability to be actived dynamically. [..]

We can control everything dynamically without self modifying code.

The _only_ point of self modifying code is performance. None other reason
to use it. lkcd is definitely called in an extremely slow path (infact
if all goes right it should never be recalled), so it doesn't give
any advantage to use self modifying code there.

> [..] That
> gives the user more fexibility. Even enterprise customers can sometimes
> hedge their bets when it comes to RAS-like features.

I agree that being able to enable/disable lkcd dynamically is fine
feature.

Andrea



-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)

[Matt D. Robinson]

richardj_moore@uk.ibm.com wrote:
> > Well, not necessarily so while lkcd is not get accepted into the standard
> > kernel source. [..]
> 
> It won't until it uses a separate driver that doesn't depend on scsi or
> ide layer.

We're working on it ... can't quit my day job, you know ... :)

--Matt
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/