LKML Archive on lore.kernel.org
 help / color / Atom feed
* [RFC] Splitting kernel headers and deprecating __KERNEL__
@ 2004-11-25 15:13 David Howells
  2004-11-25 15:23 ` David Woodhouse
                   ` (10 more replies)
  0 siblings, 11 replies; 185+ messages in thread
From: David Howells @ 2004-11-25 15:13 UTC (permalink / raw)
  To: torvalds, hch, matthew, dwmw2, aoliva; +Cc: linux-kernel, libc-hacker


We've been discussing splitting the kernel headers into userspace API headers
and kernel internal headers and deprecating the __KERNEL__ macro. This will
permit a cleaner interface between the kernel and userspace; and one that's
easier to keep up to date.

What we've come up with is this:

 (1) Create new directories in the linux sources to shadow existing include
     directories:

	NEW DIRECTORY		DIRECTORY SHADOWED
	=============		==================
	include/user/		include/linux/
	include/user-*/		include/asm-*/

     Note that this doesn't take account of the other directories under
     include/, but I don't think they're relevant.

 (2) Take each file from the shadowed directory. If it has any userspace
     relevant stuff, then:

     (a) Transfer this stuff into a file of the same name in the new
	 directory. So, for example, the syscall number list from
	 include/asm-i386/unistd.h will be transferred to
	 include/user-i386/unistd.h.

     (b) Make kernel file #include the user file. So:

		[include/asm-i386/unistd.h]
		...
		#include <user-i386/unistd.h>
		...

     (c) Where a user header file requires something from another header file
	 (such as a type), that file should include a suitable user header file
	 directly:

		[include/user-i386/termio.h]
		...
		#include <user/types.h>
		...

     (d) stdint types should be used where possible.

		[include/user-i386/termios.h]
		struct winsize {
			uint16_t ws_row;
			uint16_t ws_col;
			uint16_t ws_xpixel;
			uint16_t ws_ypixel;
		};

     (e) These header files should be bounded with __USER_XXXXX_H conditionals:

		[include/user-i386/termios.h]
		#ifndef __USER_I386_TERMIOS_H
		#define __USER_I386_TERMIOS_H
		...
		#endif /*  __USER_I386_TERMIOS_H */

 (3) Remove all #if(n)def __KERNEL__ clauses.

 (4) Remove the -D__KERNEL__ from the master kernel Makefile.

 (5) For userspace use (such as for glibc), the appropriate include/user*/
     directories should be selected and installed in /usr/include/ or wherever,
     and symlinks made. For example, on i386 arch boxes, you might find:

	SOURCE			INSTALLED AS
	======================	============
	include/user/		/usr/include/user/
	include/user-i386/	/usr/include/user-i386/
				/usr/include/linux -> user
				/usr/include/asm -> user-i386

 (6) On multi-arch archs (such as ppc64 which can also support ppc), you might
     find:

	SOURCE			INSTALLED AS
	======================	============
	include/user/		/usr/include/user/
	include/user-ppc/	/usr/include/user-ppc/
	include/user-ppc64/	/usr/include/user-ppc64/
				/usr/include/linux -> user
				/usr/include/asm-ppc -> user-ppc
				/usr/include/asm-ppc64 -> user-ppc64

     And then /usr/include/asm/ might contain files that do arch-size dependent
     switching between user-ppc and user-ppc64.


David

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-25 15:13 [RFC] Splitting kernel headers and deprecating __KERNEL__ David Howells
@ 2004-11-25 15:23 ` David Woodhouse
  2004-11-25 16:54 ` Matthew Wilcox
                   ` (9 subsequent siblings)
  10 siblings, 0 replies; 185+ messages in thread
From: David Woodhouse @ 2004-11-25 15:23 UTC (permalink / raw)
  To: David Howells; +Cc: torvalds, hch, matthew, aoliva, linux-kernel, libc-hacker

On Thu, 2004-11-25 at 15:13 +0000, David Howells wrote:
> We've been discussing splitting the kernel headers into userspace API headers
> and kernel internal headers and deprecating the __KERNEL__ macro. This will
> permit a cleaner interface between the kernel and userspace; and one that's
> easier to keep up to date.
> 
> What we've come up with is this:

I've already done something vaguely along those lines, extracting
user-visible bits from include/linux/mtd/*.h, putting them into files in
include/mtd/*.h and including them from there (except for jffs2.h which
currently does it backwards).

-- 
dwmw2


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-25 15:13 [RFC] Splitting kernel headers and deprecating __KERNEL__ David Howells
  2004-11-25 15:23 ` David Woodhouse
@ 2004-11-25 16:54 ` Matthew Wilcox
  2004-11-25 18:17   ` David Woodhouse
  2004-11-27 20:39   ` Sam Ravnborg
  2004-11-25 18:20 ` Alexandre Oliva
                   ` (8 subsequent siblings)
  10 siblings, 2 replies; 185+ messages in thread
From: Matthew Wilcox @ 2004-11-25 16:54 UTC (permalink / raw)
  To: David Howells
  Cc: torvalds, hch, matthew, dwmw2, aoliva, linux-kernel, libc-hacker

On Thu, Nov 25, 2004 at 03:13:12PM +0000, David Howells wrote:
> We've been discussing splitting the kernel headers into userspace API headers
> and kernel internal headers and deprecating the __KERNEL__ macro. This will
> permit a cleaner interface between the kernel and userspace; and one that's
> easier to keep up to date.

I am fully in support of both this idea and the current implementation
you have described below.

> What we've come up with is this:
> 
>  (1) Create new directories in the linux sources to shadow existing include
>      directories:
> 
> 	NEW DIRECTORY		DIRECTORY SHADOWED
> 	=============		==================
> 	include/user/		include/linux/
> 	include/user-*/		include/asm-*/
> 
>      Note that this doesn't take account of the other directories under
>      include/, but I don't think they're relevant.

I think they may be.  If they are, they can be accommodated.  For example:

	include/user-scsi/	include/scsi/
	include/user-net/	include/net/
	include/user-rxrpc/	include/rxrpc/

etcetera.  This only creates a conflict if someone creates a directory
foo that also clashes with asm-foo.  And if someone does propose that,
we just kill them.  Easy.

>      (d) stdint types should be used where possible.
> 
> 		[include/user-i386/termios.h]
> 		struct winsize {
> 			uint16_t ws_row;
> 			uint16_t ws_col;
> 			uint16_t ws_xpixel;
> 			uint16_t ws_ypixel;
> 		};

I really hate stdint.  Can't we use __u16 instead?

>      (e) These header files should be bounded with __USER_XXXXX_H conditionals:
> 
> 		[include/user-i386/termios.h]
> 		#ifndef __USER_I386_TERMIOS_H
> 		#define __USER_I386_TERMIOS_H
> 		...
> 		#endif /*  __USER_I386_TERMIOS_H */
> 
>  (3) Remove all #if(n)def __KERNEL__ clauses.
> 
>  (4) Remove the -D__KERNEL__ from the master kernel Makefile.
> 
>  (5) For userspace use (such as for glibc), the appropriate include/user*/
>      directories should be selected and installed in /usr/include/ or wherever,
>      and symlinks made. For example, on i386 arch boxes, you might find:
> 
> 	SOURCE			INSTALLED AS
> 	======================	============
> 	include/user/		/usr/include/user/
> 	include/user-i386/	/usr/include/user-i386/
> 				/usr/include/linux -> user
> 				/usr/include/asm -> user-i386

This proposal doesn't address the asm-generic problem directly.  Can I
presume that you intend to also create linux/include/user-generic, install
it as /usr/include/user-generic and create an asm-generic symlink that
points to user-generic?  A good problem file to be dealt with would
be asm/errno.h

>  (6) On multi-arch archs (such as ppc64 which can also support ppc), you might
>      find:
> 
> 	SOURCE			INSTALLED AS
> 	======================	============
> 	include/user/		/usr/include/user/
> 	include/user-ppc/	/usr/include/user-ppc/
> 	include/user-ppc64/	/usr/include/user-ppc64/
> 				/usr/include/linux -> user
> 				/usr/include/asm-ppc -> user-ppc
> 				/usr/include/asm-ppc64 -> user-ppc64
> 
>      And then /usr/include/asm/ might contain files that do arch-size dependent
>      switching between user-ppc and user-ppc64.

ppc might want to consider following the lead of parisc, s390 and mips
and unify at least their header files, if not their arch directory.

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-25 16:54 ` Matthew Wilcox
@ 2004-11-25 18:17   ` David Woodhouse
  2004-11-27  3:24     ` Greg KH
  2004-11-27 20:39   ` Sam Ravnborg
  1 sibling, 1 reply; 185+ messages in thread
From: David Woodhouse @ 2004-11-25 18:17 UTC (permalink / raw)
  To: Matthew Wilcox
  Cc: David Howells, torvalds, hch, aoliva, linux-kernel, libc-hacker

On Thu, 2004-11-25 at 16:54 +0000, Matthew Wilcox wrote:
> >      Note that this doesn't take account of the other directories under
> >      include/, but I don't think they're relevant.
> 
> I think they may be.  If they are, they can be accommodated.  For example:
> 
> 	include/user-scsi/	include/scsi/
> 	include/user-net/	include/net/
> 	include/user-rxrpc/	include/rxrpc/
> 
> etcetera.  This only creates a conflict if someone creates a directory
> foo that also clashes with asm-foo.  And if someone does propose that,
> we just kill them.  Easy.

Agreed.

> >      (d) stdint types should be used where possible.
> > 
> > 		[include/user-i386/termios.h]
> > 		struct winsize {
> > 			uint16_t ws_row;
> > 			uint16_t ws_col;
> > 			uint16_t ws_xpixel;
> > 			uint16_t ws_ypixel;
> > 		};
> 
> I really hate stdint.  Can't we use __u16 instead?

We're trying to clean all this crap up. I think we'd need a better
justification than 'I hate stdint' to use anything other than the
standard types which the language provides.

You'll get over it, I promise. I hated stdint too for a whole week or so
after I admitted it made sense to switch JFFS2 to it, because I actually
wanted it to be usable other than in the Linux kernel.

> This proposal doesn't address the asm-generic problem directly.  Can I
> presume that you intend to also create linux/include/user-generic, install
> it as /usr/include/user-generic and create an asm-generic symlink that
> points to user-generic?  A good problem file to be dealt with would
> be asm/errno.h

That makes sense.

> ppc might want to consider following the lead of parisc, s390 and mips
> and unify at least their header files, if not their arch directory.

That makes a certain amount of sense, although ppc64 currently lacks the
mess of platform-specific stuff that ppc32 has for various embedded
platforms. I suspect it would take a fair amount of work to produce a
single directory which everyone's happy with.

-- 
dwmw2


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-25 15:13 [RFC] Splitting kernel headers and deprecating __KERNEL__ David Howells
  2004-11-25 15:23 ` David Woodhouse
  2004-11-25 16:54 ` Matthew Wilcox
@ 2004-11-25 18:20 ` Alexandre Oliva
  2004-11-25 21:01   ` Matthew Wilcox
  2004-11-26 11:58   ` David Howells
  2004-11-25 18:45 ` Matthew Wilcox
                   ` (7 subsequent siblings)
  10 siblings, 2 replies; 185+ messages in thread
From: Alexandre Oliva @ 2004-11-25 18:20 UTC (permalink / raw)
  To: David Howells; +Cc: torvalds, hch, matthew, dwmw2, linux-kernel, libc-hacker

On Nov 25, 2004, David Howells <dhowells@redhat.com> wrote:

> 	SOURCE			INSTALLED AS
> 	======================	============
> 	include/user/		/usr/include/user/
> 	include/user-i386/	/usr/include/user-i386/
> 				/usr/include/linux -> user
> 				/usr/include/asm -> user-i386

Although user/ and user-* make a lot of sense within the kernel source
tree, I don't think these names would be very clear in /usr/include.
I'd rather use names in /usr/include that more clearly associate them
with the kernel.  Heck, even /usr/include/asm is inappropriate, but
it's been there for so long that we really shouldn't try to get rid of
it.

If I had it my way, we'd have, in the kernel tree, userland-aimed
headers in include/linux/user and include/asm-<machine>/user, and have
them installed in /usr/include/linux and /usr/include/asm-<machine>.

This means these headers shouldn't reference each other as
linux/user/something.h, but rather as linux/something.h, such that
they still work when installed in /usr/include/linux.  This may
require headers include/linux/something.h to include
linux/user/something.h, but that's already part of the proposal.

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-25 15:13 [RFC] Splitting kernel headers and deprecating __KERNEL__ David Howells
                   ` (2 preceding siblings ...)
  2004-11-25 18:20 ` Alexandre Oliva
@ 2004-11-25 18:45 ` Matthew Wilcox
  2004-11-26 11:52 ` David Howells
                   ` (6 subsequent siblings)
  10 siblings, 0 replies; 185+ messages in thread
From: Matthew Wilcox @ 2004-11-25 18:45 UTC (permalink / raw)
  To: David Howells
  Cc: torvalds, hch, matthew, dwmw2, aoliva, linux-kernel, libc-alpha

On Thu, Nov 25, 2004 at 03:13:12PM +0000, David Howells wrote:
>  (2) Take each file from the shadowed directory. If it has any userspace
>      relevant stuff, then:
> 
>      (b) Make kernel file #include the user file. So:
> 
> 		[include/asm-i386/unistd.h]
> 		...
> 		#include <user-i386/unistd.h>
> 		...

We may also want a user-asm symlink pointing to user-$ARCH.
If <linux/foo.h> wants a definition from <user-$ARCH/foo.h> then it has
to include <asm/foo.h> which includes <user-$ARCH/foo.h>.  If asm/foo.h
is empty other than the include, then it'd be nice to delete it and have
<linux/foo.h> include <user-asm/foo.h> directly.

>      (c) Where a user header file requires something from another header file
> 	 (such as a type), that file should include a suitable user header file
> 	 directly:
> 
> 		[include/user-i386/termio.h]
> 		...
> 		#include <user/types.h>
> 		...

It's occasionally been on my mind that the transition from linux -> asm
should be one way and that asm files should not include linux files.
I'm not sure this is necessarily a worthy goal, but it seems worth
mentioning.

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-25 18:20 ` Alexandre Oliva
@ 2004-11-25 21:01   ` Matthew Wilcox
  2004-11-25 22:35     ` David Woodhouse
  2004-11-26 11:47     ` Alexandre Oliva
  2004-11-26 11:58   ` David Howells
  1 sibling, 2 replies; 185+ messages in thread
From: Matthew Wilcox @ 2004-11-25 21:01 UTC (permalink / raw)
  To: Alexandre Oliva
  Cc: David Howells, torvalds, hch, matthew, dwmw2, linux-kernel, libc-alpha

On Thu, Nov 25, 2004 at 04:20:06PM -0200, Alexandre Oliva wrote:
> This means these headers shouldn't reference each other as
> linux/user/something.h, but rather as linux/something.h, such that
> they still work when installed in /usr/include/linux.  This may
> require headers include/linux/something.h to include
> linux/user/something.h, but that's already part of the proposal.

That's going to take severe brain-ache to get right ... and worse,
keep right.  These headers aren't going to get tested outside the kernel
tree often.  So we'll have missing includes and files that only work if
the <linux/> they're including is a kernel one rather than a user one.

I'm not particularly stuck on the <user/> namespace.  We could invent
a better name.  How about <kern/> and <arch/> to replace <linux/>
and <asm/>?  Obviously keeping linux/ and asm/ symlinks for backwards
compatibility.

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-25 21:01   ` Matthew Wilcox
@ 2004-11-25 22:35     ` David Woodhouse
  2004-11-26 11:42       ` Alexandre Oliva
  2004-11-27  0:13       ` Andreas Steinmetz
  2004-11-26 11:47     ` Alexandre Oliva
  1 sibling, 2 replies; 185+ messages in thread
From: David Woodhouse @ 2004-11-25 22:35 UTC (permalink / raw)
  To: Matthew Wilcox
  Cc: Alexandre Oliva, dhowells, torvalds, hch, linux-kernel, libc-alpha

On Thu, 2004-11-25 at 21:01 +0000, Matthew Wilcox wrote:
> I'm not particularly stuck on the <user/> namespace.  We could invent
> a better name.  How about <kern/> and <arch/> to replace <linux/>
> and <asm/>?  Obviously keeping linux/ and asm/ symlinks for backwards
> compatibility.

There's no _real_ need to keep them. All we need to fix is a handful of
libc implementations; anything else using them was broken anyway.

-- 
dwmw2


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-25 22:35     ` David Woodhouse
@ 2004-11-26 11:42       ` Alexandre Oliva
  2004-11-27  0:13       ` Andreas Steinmetz
  1 sibling, 0 replies; 185+ messages in thread
From: Alexandre Oliva @ 2004-11-26 11:42 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Matthew Wilcox, dhowells, torvalds, hch, linux-kernel, libc-alpha

On Nov 25, 2004, David Woodhouse <dwmw2@infradead.org> wrote:

> There's no _real_ need to keep them. All we need to fix is a handful of
> libc implementations; anything else using them was broken anyway.

It's not as simple as *fixing* them.  They'd have to go through a
transition period in which they support both formats, which probably
implies a mess of ifdefs or, worse, macro expansion in #include
directives.

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-25 21:01   ` Matthew Wilcox
  2004-11-25 22:35     ` David Woodhouse
@ 2004-11-26 11:47     ` Alexandre Oliva
  2004-11-26 11:53       ` David Woodhouse
  2004-11-27 16:12       ` Krzysztof Halasa
  1 sibling, 2 replies; 185+ messages in thread
From: Alexandre Oliva @ 2004-11-26 11:47 UTC (permalink / raw)
  To: Matthew Wilcox
  Cc: David Howells, torvalds, hch, dwmw2, linux-kernel, libc-alpha

On Nov 25, 2004, Matthew Wilcox <matthew@wil.cx> wrote:

> On Thu, Nov 25, 2004 at 04:20:06PM -0200, Alexandre Oliva wrote:
>> This means these headers shouldn't reference each other as
>> linux/user/something.h, but rather as linux/something.h, such that
>> they still work when installed in /usr/include/linux.  This may
>> require headers include/linux/something.h to include
>> linux/user/something.h, but that's already part of the proposal.

> That's going to take severe brain-ache to get right ... and worse,
> keep right.  These headers aren't going to get tested outside the kernel
> tree often.  So we'll have missing includes and files that only work if
> the <linux/> they're including is a kernel one rather than a user one.

How about moving the internals (i.e., what's not to be exported to
userland) from linux and asm elsewhere, then?

Sure, it means significantly more churn in the kernel, but there's
going to be a lot of moving stuff around one way or the other.

While at that, we could also split what's kernel internal for real and
what's to be visible to external kernel modules as well.  So we'd have
3 layers of headers, instead of two.  I'm not sure this actually makes
any sense though, since there might be lots of dependencies of headers
for modules on internal headers.

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__ 
  2004-11-25 15:13 [RFC] Splitting kernel headers and deprecating __KERNEL__ David Howells
                   ` (3 preceding siblings ...)
  2004-11-25 18:45 ` Matthew Wilcox
@ 2004-11-26 11:52 ` David Howells
  2004-11-26 23:56 ` Grzegorz Kulewski
                   ` (5 subsequent siblings)
  10 siblings, 0 replies; 185+ messages in thread
From: David Howells @ 2004-11-26 11:52 UTC (permalink / raw)
  To: Matthew Wilcox; +Cc: torvalds, hch, dwmw2, aoliva, linux-kernel, libc-hacker


> This proposal doesn't address the asm-generic problem directly.  Can I
> presume that you intend to also create linux/include/user-generic, install
> it as /usr/include/user-generic and create an asm-generic symlink that
> points to user-generic?  A good problem file to be dealt with would
> be asm/errno.h

Agreed. I'd forgotten about that.

David

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-26 11:47     ` Alexandre Oliva
@ 2004-11-26 11:53       ` David Woodhouse
  2004-11-27 16:12       ` Krzysztof Halasa
  1 sibling, 0 replies; 185+ messages in thread
From: David Woodhouse @ 2004-11-26 11:53 UTC (permalink / raw)
  To: Alexandre Oliva
  Cc: Matthew Wilcox, David Howells, torvalds, hch, linux-kernel, libc-alpha

On Fri, 2004-11-26 at 09:47 -0200, Alexandre Oliva wrote:
> On Nov 25, 2004, Matthew Wilcox <matthew@wil.cx> wrote:
> 
> > On Thu, Nov 25, 2004 at 04:20:06PM -0200, Alexandre Oliva wrote:
> >> This means these headers shouldn't reference each other as
> >> linux/user/something.h, but rather as linux/something.h, such that
> >> they still work when installed in /usr/include/linux.  This may
> >> require headers include/linux/something.h to include
> >> linux/user/something.h, but that's already part of the proposal.
> 
> > That's going to take severe brain-ache to get right ... and worse,
> > keep right.  These headers aren't going to get tested outside the kernel
> > tree often.  So we'll have missing includes and files that only work if
> > the <linux/> they're including is a kernel one rather than a user one.
> 
> How about moving the internals (i.e., what's not to be exported to
> userland) from linux and asm elsewhere, then?
> 
> Sure, it means significantly more churn in the kernel, but there's
> going to be a lot of moving stuff around one way or the other.

Not really. The kernel code was what was _allowed_ to be using those
headers with those names in the first place; it was userspace which
shouldn't necessarily have been doing so.

> While at that, we could also split what's kernel internal for real and
> what's to be visible to external kernel modules as well.  So we'd have
> 3 layers of headers, instead of two.  I'm not sure this actually makes
> any sense though, since there might be lots of dependencies of headers
> for modules on internal headers.

All modules will be entirely dependent on the full set of kernel
headers. Let's not go there.

-- 
dwmw2


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__ 
  2004-11-25 18:20 ` Alexandre Oliva
  2004-11-25 21:01   ` Matthew Wilcox
@ 2004-11-26 11:58   ` David Howells
  2004-11-26 12:00     ` David Woodhouse
  2004-11-29 16:34     ` Alexandre Oliva
  1 sibling, 2 replies; 185+ messages in thread
From: David Howells @ 2004-11-26 11:58 UTC (permalink / raw)
  To: Matthew Wilcox
  Cc: Alexandre Oliva, torvalds, hch, dwmw2, linux-kernel, libc-alpha


> I'm not particularly stuck on the <user/> namespace.  We could invent
> a better name.  How about <kern/> and <arch/> to replace <linux/>
> and <asm/>?  Obviously keeping linux/ and asm/ symlinks for backwards
> compatibility.

I'd rather not change linux and asm within the kernel itself. We'd have to
patch pretty much every file and move all the include files around. That's a
lot of patch.

How about calling the interface headers "kapi*/" instead of "user*/". In case
you haven't guessed, "kapi" would be short for "kernel-api".

Then we'd have in the kernel:

	NEW DIRECTORY		DIRECTORY SHADOWED
	=============		==================
	include/kapi/		include/linux/
	include/kapi-*/		include/asm-*/
	include/kapi-rxrpc/	include/rxrpc/
	include/kapi-net/	include/net/

And in userspace:

	SOURCE			INSTALLED AS
	======================	============
	include/kapi/		/usr/include/kapi/
	include/kapi-i386/	/usr/include/kapi-i386/
	include/kapi-generic/	/usr/include/kapi-generic/
				/usr/include/linux -> kapi
				/usr/include/asm -> kapi-i386
				/usr/include/asm-generic -> kapi-generic (?)

David

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-26 11:58   ` David Howells
@ 2004-11-26 12:00     ` David Woodhouse
  2004-11-26 14:19       ` Matthew Wilcox
  2004-11-29 16:34     ` Alexandre Oliva
  1 sibling, 1 reply; 185+ messages in thread
From: David Woodhouse @ 2004-11-26 12:00 UTC (permalink / raw)
  To: David Howells
  Cc: Matthew Wilcox, Alexandre Oliva, torvalds, hch, linux-kernel, libc-alpha

On Fri, 2004-11-26 at 11:58 +0000, David Howells wrote:
> How about calling the interface headers "kapi*/" instead of "user*/". In case
> you haven't guessed, "kapi" would be short for "kernel-api".

I don't think that change really makes any difference. The nomenclature
really isn't _that_ important.

-- 
dwmw2


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-26 12:00     ` David Woodhouse
@ 2004-11-26 14:19       ` Matthew Wilcox
  2004-11-26 14:33         ` David Woodhouse
                           ` (2 more replies)
  0 siblings, 3 replies; 185+ messages in thread
From: Matthew Wilcox @ 2004-11-26 14:19 UTC (permalink / raw)
  To: David Woodhouse
  Cc: David Howells, Matthew Wilcox, Alexandre Oliva, torvalds, hch,
	linux-kernel, libc-alpha

On Fri, Nov 26, 2004 at 12:00:43PM +0000, David Woodhouse wrote:
> On Fri, 2004-11-26 at 11:58 +0000, David Howells wrote:
> > How about calling the interface headers "kapi*/" instead of "user*/". In case
> > you haven't guessed, "kapi" would be short for "kernel-api".
> 
> I don't think that change really makes any difference. The nomenclature
> really isn't _that_ important.

Indeed.  We could also make this transparent to userspace by using a script
to copy the user-* headers to /usr/include.  Something like this:

install_user_headers:
	$(srctree)/scripts/install_user_headers.sh

#!/bin/bash
rm -rf /usr/include/asm /usr/include/linux /usr/include/asm-generic
mkdir /usr/include/asm /usr/include/linux /usr/include/asm-generic

to_user() {
	sed	-e 's,<user/,<linux/, \
		-e 's,<user-generic/,<asm-generic,' \
		-e 's,<user-[^/]*/,<asm,' \
		<$1 >$2
}

for src in include/user/*; do
	to_user $src /usr/include/linux/`basename $src`
done

for src in include/user-$ARCH/*; do
	to_user $src /usr/include/asm/`basename $src`
done

for src in include/user-generic/*; do
	to_user $src /usr/include/asm-generic/`basename $src`
done


If we really wanted to get fancy, we could also sed __u32 to uint32_t.
But that would probably cause more pain, confusion, hurt and bad feeling
than I'd ever want to be responsible for.

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-26 14:19       ` Matthew Wilcox
@ 2004-11-26 14:33         ` David Woodhouse
  2004-11-27 12:10           ` Alexander Stohr
  2004-11-27  3:46         ` Adam Heath
  2004-11-30 12:18         ` Horst von Brand
  2 siblings, 1 reply; 185+ messages in thread
From: David Woodhouse @ 2004-11-26 14:33 UTC (permalink / raw)
  To: Matthew Wilcox
  Cc: David Howells, Alexandre Oliva, torvalds, hch, linux-kernel, libc-alpha

On Fri, 2004-11-26 at 14:19 +0000, Matthew Wilcox wrote:
> On Fri, Nov 26, 2004 at 12:00:43PM +0000, David Woodhouse wrote:
> > On Fri, 2004-11-26 at 11:58 +0000, David Howells wrote:
> > > How about calling the interface headers "kapi*/" instead of "user*/". In case
> > > you haven't guessed, "kapi" would be short for "kernel-api".
> > 
> > I don't think that change really makes any difference. The nomenclature
> > really isn't _that_ important.
> 
> Indeed.  We could also make this transparent to userspace by using a script
> to copy the user-* headers to /usr/include.  Something like this:

Indeed.

> If we really wanted to get fancy, we could also sed __u32 to uint32_t.
> But that would probably cause more pain, confusion, hurt and bad feeling
> than I'd ever want to be responsible for.

Also true. Let's just use the standard types in the first place and not
screw around with having to fix it up later.

-- 
dwmw2


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-25 15:13 [RFC] Splitting kernel headers and deprecating __KERNEL__ David Howells
                   ` (4 preceding siblings ...)
  2004-11-26 11:52 ` David Howells
@ 2004-11-26 23:56 ` Grzegorz Kulewski
  2004-11-27  1:27   ` Tomas Carnecky
  2004-11-27  3:07 ` Kyle Moffett
                   ` (4 subsequent siblings)
  10 siblings, 1 reply; 185+ messages in thread
From: Grzegorz Kulewski @ 2004-11-26 23:56 UTC (permalink / raw)
  To: David Howells
  Cc: torvalds, hch, matthew, dwmw2, aoliva, linux-kernel, libc-hacker

On Thu, 25 Nov 2004, David Howells wrote:

>     (b) Make kernel file #include the user file.

Does kernel really need to include user headers? When it is definition of 
some const then it should be defined in one file (to be sure it has only 
one definition). But user headers may have some compatibility hacks that 
kernel do not need (and even maybe does not want) to have.

How you will handle that?


Thanks,

Grzegorz Kulewski


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-25 22:35     ` David Woodhouse
  2004-11-26 11:42       ` Alexandre Oliva
@ 2004-11-27  0:13       ` Andreas Steinmetz
  2004-11-27  0:26         ` David Woodhouse
  1 sibling, 1 reply; 185+ messages in thread
From: Andreas Steinmetz @ 2004-11-27  0:13 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Matthew Wilcox, Alexandre Oliva, dhowells, torvalds, hch,
	linux-kernel, libc-alpha

David Woodhouse wrote:
> On Thu, 2004-11-25 at 21:01 +0000, Matthew Wilcox wrote:
> 
>>I'm not particularly stuck on the <user/> namespace.  We could invent
>>a better name.  How about <kern/> and <arch/> to replace <linux/>
>>and <asm/>?  Obviously keeping linux/ and asm/ symlinks for backwards
>>compatibility.
> 
> 
> There's no _real_ need to keep them. All we need to fix is a handful of
> libc implementations; anything else using them was broken anyway.
> 
<rant mode>

Cute idea. So any new definition which is supposed to be used in 
userspace will take at least a year to propagate - well, I know, one can 
always look for the definition in the kernel headers, copy it, ...
Now this is no solution.

And how do you want to deal with the fact that up to now all the 
netfilter headers required for userspace programming live in the kernel 
include tree? Now this has been like this for quite some years. Shall 
one no longer use netfilter?

It is easy to be happy with the status quo and to reject any attempt to 
change things for the better. If you really wanted to fix (g)libc you 
would have taken action long ago. You're around for long enough.

</rant mode>
-- 
Andreas Steinmetz                       SPAMmers use robotrap@domdv.de

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27  0:13       ` Andreas Steinmetz
@ 2004-11-27  0:26         ` David Woodhouse
  2004-11-27  0:30           ` Andreas Steinmetz
  0 siblings, 1 reply; 185+ messages in thread
From: David Woodhouse @ 2004-11-27  0:26 UTC (permalink / raw)
  To: Andreas Steinmetz
  Cc: Matthew Wilcox, Alexandre Oliva, dhowells, torvalds, hch,
	linux-kernel, libc-alpha

On Sat, 2004-11-27 at 01:13 +0100, Andreas Steinmetz wrote:
> And how do you want to deal with the fact that up to now all the 
> netfilter headers required for userspace programming live in the kernel 
> include tree? Now this has been like this for quite some years. Shall 
> one no longer use netfilter?

Don't get me started on fucking netfilter. Have you tried running 32-bit
userspace on a 64-bit kernel recently? Throw away the fucking netfilter
shite and burn it.

-- 
dwmw2



^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27  0:26         ` David Woodhouse
@ 2004-11-27  0:30           ` Andreas Steinmetz
  2004-11-27  0:38             ` David Woodhouse
  0 siblings, 1 reply; 185+ messages in thread
From: Andreas Steinmetz @ 2004-11-27  0:30 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Matthew Wilcox, Alexandre Oliva, dhowells, torvalds, hch,
	linux-kernel, libc-alpha

David Woodhouse wrote:
> On Sat, 2004-11-27 at 01:13 +0100, Andreas Steinmetz wrote:
> 
>>And how do you want to deal with the fact that up to now all the 
>>netfilter headers required for userspace programming live in the kernel 
>>include tree? Now this has been like this for quite some years. Shall 
>>one no longer use netfilter?
> 
> 
> Don't get me started on fucking netfilter. Have you tried running 32-bit
> userspace on a 64-bit kernel recently? Throw away the fucking netfilter
> shite and burn it.
> 

I know very well about this problem. Still, you can easily use 64-bit 
applications with regards to netfilter. If you are so annoyed about this 
problem I would suggest you contact the netfilter developers to find a 
solution :-)

-- 
Andreas Steinmetz                       SPAMmers use robotrap@domdv.de

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27  0:30           ` Andreas Steinmetz
@ 2004-11-27  0:38             ` David Woodhouse
  0 siblings, 0 replies; 185+ messages in thread
From: David Woodhouse @ 2004-11-27  0:38 UTC (permalink / raw)
  To: Andreas Steinmetz
  Cc: Matthew Wilcox, Alexandre Oliva, dhowells, torvalds, hch,
	linux-kernel, libc-alpha

On Sat, 2004-11-27 at 01:30 +0100, Andreas Steinmetz wrote:
> I know very well about this problem. Still, you can easily use 64-bit 
> applications with regards to netfilter. If you are so annoyed about this 
> problem I would suggest you contact the netfilter developers to find a 
> solution :-)

It's a shame _they_ didn't know well about this problem five or so years
ago when the rest of us did. Oh well, it's on the list of things to
attempt when I can actually stomach it.

In the meantime, don't even think about claiming that netfilter is an
example of _anything_ which you can hold up as an example when we're
talking about cleaning stuff up. Breaking it in its current state would
be a feature, not a bug.

-- 
dwmw2



^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-26 23:56 ` Grzegorz Kulewski
@ 2004-11-27  1:27   ` Tomas Carnecky
  2004-11-27  1:41     ` Grzegorz Kulewski
  0 siblings, 1 reply; 185+ messages in thread
From: Tomas Carnecky @ 2004-11-27  1:27 UTC (permalink / raw)
  To: Grzegorz Kulewski
  Cc: David Howells, torvalds, hch, matthew, dwmw2, aoliva,
	linux-kernel, libc-hacker

Grzegorz Kulewski wrote:
> On Thu, 25 Nov 2004, David Howells wrote:
> 
>>     (b) Make kernel file #include the user file.
> 
> 
> Does kernel really need to include user headers? When it is definition
 > of some const then it should be defined in one file (to be sure it has
 > only one definition).

You have do define a interface between the kernel and the userspace..
you either include kernel headers from userspace (with a lot of 
__KERNEL__ in them) or you make separate headers with the definitions 
and include them in both kernel and userspace (better).
BTW, these are not userspace headers like the ones in /usr/include, 
those are just special headers preparated so that they can be included 
both from the kernel and userspace.

> But user headers may have some compatibility hacks
> that kernel do not need (and even maybe does not want) to have.

About the compatibility hacks.. now it's time to remove them, together 
with this change. I don't think this will happen before 2.7/2.8 and 
until then all should have changed their code.
If you announce these changes soon enough and the developers have enough 
time to change their code, I don't see any problems.
Maybe you also could wrap these definitions in some #ifdef's and mark 
them as deprecated and write somewhere that they'll be removed in the 
next stable tree (2.8). So you could check if a library compiles with 
the new headers or if it still uses some old definitions.

tom

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27  1:27   ` Tomas Carnecky
@ 2004-11-27  1:41     ` Grzegorz Kulewski
  2004-11-27  2:03       ` Tomas Carnecky
  0 siblings, 1 reply; 185+ messages in thread
From: Grzegorz Kulewski @ 2004-11-27  1:41 UTC (permalink / raw)
  To: Tomas Carnecky
  Cc: David Howells, torvalds, hch, matthew, dwmw2, aoliva, linux-kernel

On Sat, 27 Nov 2004, Tomas Carnecky wrote:

> Grzegorz Kulewski wrote:
>> On Thu, 25 Nov 2004, David Howells wrote:
>> 
>>> (b) Make kernel file #include the user file.
>> 
>> 
>> Does kernel really need to include user headers? When it is definition
>> of some const then it should be defined in one file (to be sure it has
>> only one definition).
>
> You have do define a interface between the kernel and the userspace..
> you either include kernel headers from userspace (with a lot of __KERNEL__ in 
> them) or you make separate headers with the definitions and include them in 
> both kernel and userspace (better).
> BTW, these are not userspace headers like the ones in /usr/include, those are 
> just special headers preparated so that they can be included both from the 
> kernel and userspace.

Ok, so maybe do it in this way:
1. common headers (included by 2. and 3.)
2. kernel headers (things only for kernel + included 1.)
3. userspace headers (things only for userspace + included 1.)

This way we will have no ifdefs, one definition per thing, user code in 
userspace and kernel code in kernelspace only.


>> But user headers may have some compatibility hacks
>> that kernel do not need (and even maybe does not want) to have.
>
> About the compatibility hacks.. now it's time to remove them, together with 
> this change. I don't think this will happen before 2.7/2.8 and until then all 
> should have changed their code.
> If you announce these changes soon enough and the developers have enough time 
> to change their code, I don't see any problems.
> Maybe you also could wrap these definitions in some #ifdef's and mark them as 
> deprecated and write somewhere that they'll be removed in the next stable 
> tree (2.8). So you could check if a library compiles with the new headers or 
> if it still uses some old definitions.

Are you talking about breaking userspace (API and ABI) compatibility? And 
possibly breaking compatibility with older versions of standards? I do not 
think it could happen. (Well at least not for common widely-used APIs).

Instead we can place such userspace only hacks in 3.


Thanks,

Grzegorz Kulewski


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27  1:41     ` Grzegorz Kulewski
@ 2004-11-27  2:03       ` Tomas Carnecky
  0 siblings, 0 replies; 185+ messages in thread
From: Tomas Carnecky @ 2004-11-27  2:03 UTC (permalink / raw)
  To: Grzegorz Kulewski
  Cc: David Howells, torvalds, hch, matthew, dwmw2, aoliva, linux-kernel

Grzegorz Kulewski wrote:
> Ok, so maybe do it in this way:
> 1. common headers (included by 2. and 3.)
> 2. kernel headers (things only for kernel + included 1.)
> 3. userspace headers (things only for userspace + included 1.)

Are there really any definitions that belong _only_ to userspace?
AFAIK all that is in the kernel sources is used by the kernel, so
there are no headers that would fit into your category 3. There is
no reason to put such things into the kernel sources.

> Are you talking about breaking userspace (API and ABI) compatibility? 
> And possibly breaking compatibility with older versions of standards? I 
> do not think it could happen. (Well at least not for common widely-used 
> APIs).
> 
> Instead we can place such userspace only hacks in 3.

The API can be changed, I don't see any problems with the API
compatibility, if you document all changes then it's no problem for
the developers to change their code.
And the ABI? Well, I compile my kernel by myself, as I do all my 
userspace (gentoo). And the distributions still could define
__DEPRECATED__ (or something similar).

I mean.. come on.. how hard is it to change some lines of code,
some structure names, some function arguments.
I know that there's a problem if you have binary only files, but
I doubt that anyone is running a exe that was compiled for against
kernel 2.0 and every big company compiles their projects against each
stable kernel line.
Oh.. and I forgot, you usually don't include the kernel headers in your
'hello world' program, most of the applications use (g)libc, one big
library, so you'd need to change only very few projects, although I
admit that can be rather big projects, but still...

BTW, how much smaller would the kernel be if we removed all the
backwards compatibility? I mean the size of the compiled kernel
image.

tom

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-25 15:13 [RFC] Splitting kernel headers and deprecating __KERNEL__ David Howells
                   ` (5 preceding siblings ...)
  2004-11-26 23:56 ` Grzegorz Kulewski
@ 2004-11-27  3:07 ` Kyle Moffett
  2004-11-27  4:29 ` Tonnerre
                   ` (3 subsequent siblings)
  10 siblings, 0 replies; 185+ messages in thread
From: Kyle Moffett @ 2004-11-27  3:07 UTC (permalink / raw)
  To: David Howells
  Cc: aoliva, linux-kernel, matthew, dwmw2, torvalds, libc-hacker, hch

On Nov 25, 2004, at 10:13, David Howells wrote:
> We've been discussing splitting the kernel headers into userspace API 
> headers
> and kernel internal headers and deprecating the __KERNEL__ macro. This 
> will
> permit a cleaner interface between the kernel and userspace; and one 
> that's
> easier to keep up to date.

Yay!!! Finally!!!

> 	NEW DIRECTORY		DIRECTORY SHADOWED
> 	=============		==================
> 	include/user/		include/linux/
> 	include/user-*/		include/asm-*/

How about include/abi for the platform-independent headers and 
include/arch-*
for the platform-specific headers?  Then include/arch would be 
symlinked to the
correct arch-* directory.

>      Note that this doesn't take account of the other directories under
>      include/, but I don't think they're relevant.

Perhaps such directories should either be broken out into 
abi/foo<->linux/foo
pieces, or put into their own top-level directories.

>  (2) Take each file from the shadowed directory. If it has any 
> userspace
>      relevant stuff, then:
[...snip...]

Something like this would warn about incorrect uses of the linux/ 
headers, yet
still allow for a period of backwards compatibility.  Once such period 
has passed,
then linux/* would not even be installed, only abi/* and arch/*

include/linux/foo.h:
#ifndef _LINUX_FOO_H
#define _LINUX_FOO_H 1

#ifndef __KERNEL__
# warning "Directly including <kernel/foo.h> is deprecated!!!"
# warning "Please change your program to include <abi/foo.h> instead!!!"
#endif

#include <abi/foo.h>

#ifdef __KERNEL__
/* Old __KERNEL__ contents of include/linux/foo.h */
#endif

#endif /* not _LINUX_FOO_H */


include/abi/foo.h:
#ifndef _ABI_FOO_H
#define_ABI_FOO_H 1

#include <abi/bar.h>
#include <arch/baz.h>

/* Old non-__KERNEL__ contents of include/linux/foo.h */

#endif /* not _ABI_FOO_H */

Cheers,
Kyle Moffett

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCM/CS/IT/U d- s++: a17 C++++>$ UB/L/X/*++++(+)>$ P+++(++++)>$
L++++(+++) E W++(+) N+++(++) o? K? w--- O? M++ V? PS+() PE+(-) Y+
PGP+++ t+(+++) 5 X R? tv-(--) b++++(++) DI+ D+ G e->++++$ h!*()>++$ r  
!y?(-)
------END GEEK CODE BLOCK------



^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-25 18:17   ` David Woodhouse
@ 2004-11-27  3:24     ` Greg KH
  2004-11-28 23:37       ` Paul Mackerras
  0 siblings, 1 reply; 185+ messages in thread
From: Greg KH @ 2004-11-27  3:24 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Matthew Wilcox, David Howells, torvalds, hch, aoliva,
	linux-kernel, libc-hacker

On Thu, Nov 25, 2004 at 06:17:42PM +0000, David Woodhouse wrote:
> On Thu, 2004-11-25 at 16:54 +0000, Matthew Wilcox wrote:
> > >      (d) stdint types should be used where possible.
> > > 
> > > 		[include/user-i386/termios.h]
> > > 		struct winsize {
> > > 			uint16_t ws_row;
> > > 			uint16_t ws_col;
> > > 			uint16_t ws_xpixel;
> > > 			uint16_t ws_ypixel;
> > > 		};
> > 
> > I really hate stdint.  Can't we use __u16 instead?
> 
> We're trying to clean all this crap up. I think we'd need a better
> justification than 'I hate stdint' to use anything other than the
> standard types which the language provides.

The justification is that it doesn't properly describe the variable size
(think userspace 32 bit variable vs. kernelspace 32 bit variable.)  We
need to stick with the proper __u32 type variables for data that crosses
the userspace/kernelspace boundry.

See Linus's posts about this topic on lkml in the past for more details
about this.

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-26 14:19       ` Matthew Wilcox
  2004-11-26 14:33         ` David Woodhouse
@ 2004-11-27  3:46         ` Adam Heath
  2004-11-30 12:18         ` Horst von Brand
  2 siblings, 0 replies; 185+ messages in thread
From: Adam Heath @ 2004-11-27  3:46 UTC (permalink / raw)
  To: Matthew Wilcox; +Cc: linux-kernel

On Fri, 26 Nov 2004, Matthew Wilcox wrote:

> #!/bin/bash

<nitpick>
This is not a bash script.
</nitpick>

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27  4:29 ` Tonnerre
@ 2004-11-27  3:51   ` Matthew Wilcox
  2004-11-27 16:47     ` Randy.Dunlap
  2004-11-27  4:05   ` Chris Friesen
  2004-11-27  4:30   ` Tomas Carnecky
  2 siblings, 1 reply; 185+ messages in thread
From: Matthew Wilcox @ 2004-11-27  3:51 UTC (permalink / raw)
  To: Tonnerre
  Cc: David Howells, torvalds, hch, matthew, dwmw2, aoliva,
	linux-kernel, libc-hacker

On Sat, Nov 27, 2004 at 05:29:42AM +0100, Tonnerre wrote:
> Fnord alert: you're trying to change the API and the ABI of Linux. The thing

No we aren't.  We're just trying to make the kernel interface more
explicit.  Nothing should change as far as userspace programs are
concerned.  It might need some coordination with the various libcs.

And enough with the doom-and-gloom big-companies-are-scared approach, OK?
It's (a) untrue, (b) irrelevant, and (c) many of us work for big companies
that are doing Linux already.  We don't need to be lectured by you.

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27  4:29 ` Tonnerre
  2004-11-27  3:51   ` Matthew Wilcox
@ 2004-11-27  4:05   ` Chris Friesen
  2004-11-27  4:30   ` Tomas Carnecky
  2 siblings, 0 replies; 185+ messages in thread
From: Chris Friesen @ 2004-11-27  4:05 UTC (permalink / raw)
  To: Tonnerre
  Cc: David Howells, torvalds, hch, matthew, dwmw2, aoliva,
	linux-kernel, libc-hacker

Tonnerre wrote:

> Fnord alert: you're trying to change the API and the ABI of Linux.

No he's not.  He's changing the location where the API is defined, and the ABI 
doesn't change a bit.

Existing binaries will continue to run without modification.

Chris

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-25 15:13 [RFC] Splitting kernel headers and deprecating __KERNEL__ David Howells
                   ` (6 preceding siblings ...)
  2004-11-27  3:07 ` Kyle Moffett
@ 2004-11-27  4:29 ` Tonnerre
  2004-11-27  3:51   ` Matthew Wilcox
                     ` (2 more replies)
  2004-11-27 21:03 ` Sam Ravnborg
                   ` (2 subsequent siblings)
  10 siblings, 3 replies; 185+ messages in thread
From: Tonnerre @ 2004-11-27  4:29 UTC (permalink / raw)
  To: David Howells
  Cc: torvalds, hch, matthew, dwmw2, aoliva, linux-kernel, libc-hacker

Salut,

On Thu, Nov 25, 2004 at 03:13:12PM +0000, David Howells wrote:
> We've been discussing splitting the kernel headers into userspace API headers
> and kernel internal headers and deprecating the __KERNEL__ macro. This will
> permit a cleaner interface between the kernel and userspace; and one that's
> easier to keep up to date.

Fnord alert: you're trying to change the API and the ABI of Linux. The thing
you fail to see is that Linux is not the only operating system in the world. 
Big companies are very prone of using their compatibility code to make their
program run in any odd way they like without actually having to change
anything. Also, you're talking about assumptions that have been true for
decades.

Big companies are already eyeing Linux with fear because our kernels don't
actually work and because we break a lot of compatibility in the stable
series. If you now come up with something that breaks basically everything
so they need a new libc and everything, it's rather likely that they'll run
away.

I just want you to consider that. Linux needs to get to a stable state of
affairs at some time.

			    Tonnerre


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27  4:29 ` Tonnerre
  2004-11-27  3:51   ` Matthew Wilcox
  2004-11-27  4:05   ` Chris Friesen
@ 2004-11-27  4:30   ` Tomas Carnecky
  2 siblings, 0 replies; 185+ messages in thread
From: Tomas Carnecky @ 2004-11-27  4:30 UTC (permalink / raw)
  To: Tonnerre
  Cc: David Howells, torvalds, hch, matthew, dwmw2, aoliva,
	linux-kernel, libc-hacker

Tonnerre wrote:
> Salut,
> 
> On Thu, Nov 25, 2004 at 03:13:12PM +0000, David Howells wrote:
> 
>>We've been discussing splitting the kernel headers into userspace API headers
>>and kernel internal headers and deprecating the __KERNEL__ macro. This will
>>permit a cleaner interface between the kernel and userspace; and one that's
>>easier to keep up to date.
> 
> 
> Fnord alert: you're trying to change the API and the ABI of Linux. The thing
> you fail to see is that Linux is not the only operating system in the world. 
> Big companies are very prone of using their compatibility code to make their
> program run in any odd way they like without actually having to change
> anything. Also, you're talking about assumptions that have been true for
> decades.
> 
> Big companies are already eyeing Linux with fear because our kernels don't
> actually work and because we break a lot of compatibility in the stable
> series. If you now come up with something that breaks basically everything
> so they need a new libc and everything, it's rather likely that they'll run
> away.
> 

I didn't say to change this now, not tomorrow...
Just tell them to change their code because, somewhere in the future,
the API might change. Maybe the new kernel APIs are better, faster, more
secure, so perhaps there is a good reason as why it would be better to
use the new API.
And for those who want to use the old API, let's give them the ability,
but also make it possible to compile the kernel with _only_ the new API.
The different distributions will take care of it.

And about the compatibility layer.. the companies have to make a
different one for each stable series anyways.


tom

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-26 14:33         ` David Woodhouse
@ 2004-11-27 12:10           ` Alexander Stohr
  0 siblings, 0 replies; 185+ messages in thread
From: Alexander Stohr @ 2004-11-27 12:10 UTC (permalink / raw)
  To: David Woodhouse, Matthew Wilcox
  Cc: David Howells, Alexandre Oliva, torvalds, hch, linux-kernel, libc-alpha

> Matthew Wilcox wrote:
> > Indeed.  We could also make this transparent to userspace by using a script
> > to copy the user-* headers to /usr/include.  Something like this:

some administrator would like to only create symlinks for saving disk space.

others would like a true "install" with copying files so that they can delete
the kernel sources anytime they do want.

for me i would install all those kernel realted files into 
the well known /lib/modules/<kernel-version>. 
so its a no question that a single kernel install 
can be cleaned up by deleting those dir recursively.

the only part remaining somewhere else in the filesystem
would then be /boot/bzImage since booting is a special case.
there are several architectures where the bzImage is be able 
to reside in /lib/modules/... but there are others archs as well.

David Woodhouse wrote:
> Matthew Wilcox wrote:
> > If we really wanted to get fancy, we could also sed __u32 to uint32_t.
> > But that would probably cause more pain, confusion, hurt and bad feeling
> > than I'd ever want to be responsible for.
> 
> Also true. Let's just use the standard types in the first place and not
> screw around with having to fix it up later.

uint32_t is the type you should get when using the standarized statement
  #include <stdint.h>
Am i right? Does that header belong to the international C99 Standard?
So you are suggesting the kernel interface should stick to that standard?

Changing it that way would hit a noticeable amount of non kernel code.
Lets consider the opposite point of view. If we do postpone such a
change to the standards for some 3 to 10 more years, then i expect
the impact on software and problem for getting it solve is bigger.

Maybe that sort of change is something qualified for beeing done 
inside a true development kernel tree like Linux 2.7.xx which might
then beeing qualified for getting a Linux 3.0.0 release kernel tree.

-Alex.


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-26 11:47     ` Alexandre Oliva
  2004-11-26 11:53       ` David Woodhouse
@ 2004-11-27 16:12       ` Krzysztof Halasa
  1 sibling, 0 replies; 185+ messages in thread
From: Krzysztof Halasa @ 2004-11-27 16:12 UTC (permalink / raw)
  To: Alexandre Oliva
  Cc: Matthew Wilcox, David Howells, torvalds, hch, dwmw2,
	linux-kernel, libc-alpha

Alexandre Oliva <aoliva@redhat.com> writes:

> How about moving the internals (i.e., what's not to be exported to
> userland) from linux and asm elsewhere, then?

That's what I proposed several {months,weeks} ago. From a technical
point of view it seems like a superior solution:

- /usr/include/{linux,asm} -> /usr/src/linux/include/{linux,asm} = no
  problems with kernel and userspace includes.
- while Linux is quite a big program, it is _one_ program = no need to
  patch the whole universe, only the kernel would need changing
  (of course broken userspace code would need fixing, but that's
  due to their brokeness and not due to the change).

> Sure, it means significantly more churn in the kernel, but there's
> going to be a lot of moving stuff around one way or the other.

No doubt. Still, quite simple automatic operation, we've seen much larger
patches.

API backward-compatibility (including C libs) is IMHO much more
important than just some kernel patch size (I realize ABI compatibility
would be preserved in all cases).

And we can do it anytime - no need for waiting for some libc change,
no need for waiting for "2.7/2.8".

> While at that, we could also split what's kernel internal for real and
> what's to be visible to external kernel modules as well.

Can't see benefits.
-- 
Krzysztof Halasa

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27  3:51   ` Matthew Wilcox
@ 2004-11-27 16:47     ` Randy.Dunlap
  2004-11-27 17:16       ` David Woodhouse
  0 siblings, 1 reply; 185+ messages in thread
From: Randy.Dunlap @ 2004-11-27 16:47 UTC (permalink / raw)
  To: Matthew Wilcox
  Cc: Tonnerre, David Howells, torvalds, hch, dwmw2, aoliva,
	linux-kernel, libc-hacker

Matthew Wilcox wrote:
> On Sat, Nov 27, 2004 at 05:29:42AM +0100, Tonnerre wrote:
> 
>>Fnord alert: you're trying to change the API and the ABI of Linux. The thing
> 
> 
> No we aren't.  We're just trying to make the kernel interface more
> explicit.  Nothing should change as far as userspace programs are
> concerned.  It might need some coordination with the various libcs.

Speaking of more explicit, there are various asm-ARCH header
files that do or do not hide (via __KERNEL__) interfaces
such as:	get_unaligned()
and the atomic operations.

So are these Linux kernel exported APIs, or do they belong
in some library?

We'll need to decide on a consistent remedy for these,
and such changes could be made at any time IMO (i.e., soon).


> And enough with the doom-and-gloom big-companies-are-scared approach, OK?
> It's (a) untrue, (b) irrelevant, and (c) many of us work for big companies
> that are doing Linux already.  We don't need to be lectured by you.

-- 
~Randy

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27 16:47     ` Randy.Dunlap
@ 2004-11-27 17:16       ` David Woodhouse
  2004-11-27 22:53         ` Arnd Bergmann
  0 siblings, 1 reply; 185+ messages in thread
From: David Woodhouse @ 2004-11-27 17:16 UTC (permalink / raw)
  To: Randy.Dunlap
  Cc: Matthew Wilcox, Tonnerre, David Howells, torvalds, hch, aoliva,
	linux-kernel, libc-hacker

On Sat, 2004-11-27 at 08:47 -0800, Randy.Dunlap wrote:
> Speaking of more explicit, there are various asm-ARCH header
> files that do or do not hide (via __KERNEL__) interfaces
> such as:	get_unaligned()
> and the atomic operations.
> 
> So are these Linux kernel exported APIs, or do they belong
> in some library?

Both of those are kernel-private and should not be visible.

-- 
dwmw2



^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-25 16:54 ` Matthew Wilcox
  2004-11-25 18:17   ` David Woodhouse
@ 2004-11-27 20:39   ` Sam Ravnborg
  1 sibling, 0 replies; 185+ messages in thread
From: Sam Ravnborg @ 2004-11-27 20:39 UTC (permalink / raw)
  To: Matthew Wilcox
  Cc: David Howells, torvalds, hch, dwmw2, aoliva, linux-kernel, libc-hacker

On Thu, Nov 25, 2004 at 04:54:33PM +0000, Matthew Wilcox wrote:
> >  (5) For userspace use (such as for glibc), the appropriate include/user*/
> >      directories should be selected and installed in /usr/include/ or wherever,
> >      and symlinks made. For example, on i386 arch boxes, you might find:
> > 
> > 	SOURCE			INSTALLED AS
> > 	======================	============
> > 	include/user/		/usr/include/user/
> > 	include/user-i386/	/usr/include/user-i386/
> > 				/usr/include/linux -> user
> > 				/usr/include/asm -> user-i386
> 
> This proposal doesn't address the asm-generic problem directly.  Can I
> presume that you intend to also create linux/include/user-generic, install
> it as /usr/include/user-generic and create an asm-generic symlink that
> points to user-generic?  A good problem file to be dealt with would
> be asm/errno.h

Do not allow asm-generic to be used direct by userspace.
The example with errno.h is handled by user-i386/errno.h which includes
asm-generic/errno.h.

Would that do the trick - yes?

	Sam

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-25 15:13 [RFC] Splitting kernel headers and deprecating __KERNEL__ David Howells
                   ` (7 preceding siblings ...)
  2004-11-27  4:29 ` Tonnerre
@ 2004-11-27 21:03 ` Sam Ravnborg
  2004-11-27 21:11   ` Andreas Steinmetz
  2004-11-29 11:08 ` David Howells
  2004-11-29 20:01 ` H. Peter Anvin
  10 siblings, 1 reply; 185+ messages in thread
From: Sam Ravnborg @ 2004-11-27 21:03 UTC (permalink / raw)
  To: David Howells
  Cc: torvalds, hch, matthew, dwmw2, aoliva, linux-kernel, libc-hacker


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #0: Type: text/plain; charset=unknown-8bit, Size: 2228 bytes --]

On Thu, Nov 25, 2004 at 03:13:12PM +0000, David Howells wrote:
> 
> We've been discussing splitting the kernel headers into userspace API headers
> and kernel internal headers and deprecating the __KERNEL__ macro. This will
> permit a cleaner interface between the kernel and userspace; and one that's
> easier to keep up to date.
> 
> What we've come up with is this:
> 
>  (1) Create new directories in the linux sources to shadow existing include
>      directories:
> 
> 	NEW DIRECTORY		DIRECTORY SHADOWED
> 	=============		==================
> 	include/user/		include/linux/
> 	include/user-*/		include/asm-*/
> 
>      Note that this doesn't take account of the other directories under
>      include/, but I don't think they're relevant.

If we go for some resturcturing of include/ then we should get rid of
the annoying asm symlink. Following layout deals with that:

include/<arch>/asm		<= Files from include/asm-<arch>
include/<arch>/mach*		<= Files from include/mach-*

This layout solve the symlink issue in an elegant way.
We need to do trivial changes to compiler options to make it work. Changing
compiler options is much more relaible than using symlinks.

Then the userspace part would then be located in:
include/<arch>/user-asm

Userspace shall not be 'allowed' to include files from asm-generic direct,
but asm-generic shall be part of the userspace api.


>  (2) Take each file from the shadowed directory. If it has any userspace
>      relevant stuff, then:
> 
>      (b) Make kernel file #include the user file. So:
> 
> 		[include/asm-i386/unistd.h]
> 		...
> 		#include <user-i386/unistd.h>
In example above it would become:
 		#include <user-asm/unistd.h>

With repsect to consistency of the userspace header files then we could define
a few simple rules:
1) Files in include/user must be able to compile standalone - so they include
   all files they need.
2) They are not allowed to use any files outside the kernel
3) Must not use __KERNEL__ in any place

Writing a small script / Makefile that does these checks should be trivial.
Jörn wrote something similar for include/linux at one point in time.
If it is fast then we just let it be part of the standard compile so it is
checked often.

	Sam

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27 21:03 ` Sam Ravnborg
@ 2004-11-27 21:11   ` Andreas Steinmetz
  2004-11-27 21:19     ` Sam Ravnborg
  0 siblings, 1 reply; 185+ messages in thread
From: Andreas Steinmetz @ 2004-11-27 21:11 UTC (permalink / raw)
  To: Sam Ravnborg
  Cc: David Howells, torvalds, hch, matthew, dwmw2, aoliva,
	linux-kernel, libc-hacker

Sam Ravnborg wrote:
> On Thu, Nov 25, 2004 at 03:13:12PM +0000, David Howells wrote:
> 
>>We've been discussing splitting the kernel headers into userspace API headers
>>and kernel internal headers and deprecating the __KERNEL__ macro. This will
>>permit a cleaner interface between the kernel and userspace; and one that's
>>easier to keep up to date.
>>
>>What we've come up with is this:
>>
>> (1) Create new directories in the linux sources to shadow existing include
>>     directories:
>>
>>	NEW DIRECTORY		DIRECTORY SHADOWED
>>	=============		==================
>>	include/user/		include/linux/
>>	include/user-*/		include/asm-*/
>>
>>     Note that this doesn't take account of the other directories under
>>     include/, but I don't think they're relevant.
> 
> 
> If we go for some resturcturing of include/ then we should get rid of
> the annoying asm symlink. Following layout deals with that:
> 
> include/<arch>/asm		<= Files from include/asm-<arch>
> include/<arch>/mach*		<= Files from include/mach-*
> 
> This layout solve the symlink issue in an elegant way.
> We need to do trivial changes to compiler options to make it work. Changing
> compiler options is much more relaible than using symlinks.
> 
> Then the userspace part would then be located in:
> include/<arch>/user-asm
> 

This complicates things for bi-arch architectures like x86_64 where one 
can use a dispatcher directory instead of a symlink to suit include/asm 
for 32bit as well as 64bit.

-- 
Andreas Steinmetz                       SPAMmers use robotrap@domdv.de

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27 21:11   ` Andreas Steinmetz
@ 2004-11-27 21:19     ` Sam Ravnborg
  2004-11-27 21:49       ` Andreas Steinmetz
  0 siblings, 1 reply; 185+ messages in thread
From: Sam Ravnborg @ 2004-11-27 21:19 UTC (permalink / raw)
  To: Andreas Steinmetz
  Cc: Sam Ravnborg, David Howells, torvalds, hch, matthew, dwmw2,
	aoliva, linux-kernel, libc-hacker

On Sat, Nov 27, 2004 at 10:11:43PM +0100, Andreas Steinmetz wrote:
> >If we go for some resturcturing of include/ then we should get rid of
> >the annoying asm symlink. Following layout deals with that:
> >
> >include/<arch>/asm		<= Files from include/asm-<arch>
> >include/<arch>/mach*		<= Files from include/mach-*
> >
> >This layout solve the symlink issue in an elegant way.
> >We need to do trivial changes to compiler options to make it work. Changing
> >compiler options is much more relaible than using symlinks.
> >
> >Then the userspace part would then be located in:
> >include/<arch>/user-asm
> >
> 
> This complicates things for bi-arch architectures like x86_64 where one 
> can use a dispatcher directory instead of a symlink to suit include/asm 
> for 32bit as well as 64bit.
X86_64 does not create any special symlinks todays so I do not see the point?
And still a symlink is just the wrong way to solve the problem.
Adjusting options to the compiler is the way to do it.

	Sam

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27 21:19     ` Sam Ravnborg
@ 2004-11-27 21:49       ` Andreas Steinmetz
  2004-11-27 23:21         ` Arnd Bergmann
  0 siblings, 1 reply; 185+ messages in thread
From: Andreas Steinmetz @ 2004-11-27 21:49 UTC (permalink / raw)
  To: Sam Ravnborg
  Cc: David Howells, torvalds, hch, matthew, dwmw2, aoliva,
	linux-kernel, libc-hacker

Sam Ravnborg wrote:
> On Sat, Nov 27, 2004 at 10:11:43PM +0100, Andreas Steinmetz wrote:
> 
>>>If we go for some resturcturing of include/ then we should get rid of
>>>the annoying asm symlink. Following layout deals with that:
>>>
>>>include/<arch>/asm		<= Files from include/asm-<arch>
>>>include/<arch>/mach*		<= Files from include/mach-*
>>>
>>>This layout solve the symlink issue in an elegant way.
>>>We need to do trivial changes to compiler options to make it work. Changing
>>>compiler options is much more relaible than using symlinks.
>>>
>>>Then the userspace part would then be located in:
>>>include/<arch>/user-asm
>>>
>>
>>This complicates things for bi-arch architectures like x86_64 where one 
>>can use a dispatcher directory instead of a symlink to suit include/asm 
>>for 32bit as well as 64bit.
> 
> X86_64 does not create any special symlinks todays so I do not see the point?
> And still a symlink is just the wrong way to solve the problem.
> Adjusting options to the compiler is the way to do it.

Ok, this is not in the kernel, but think of the following: an 
application needs and include/asm header. The application may be 
compiled as 32 bit and as 64 bit. Then you could use a real asm 
directory instead of a symlink and use, e.g. for asm/byteorder.h:

#ifdef __x86_64__
#include <asm-x86_64/byteorder.h>
#else
#include <asm-i386/byteorder.h>
#endif

This way both archs are satisfied from a userspace point of view without 
  having to take care if the source is compiled on 32bit or 64bit. If 
you kill the symlink this neat dispatcher can't really be done anymore 
that easy.
It is not necessary to do anything in the kernel tree here except to 
keep the symlink to allow for this kind of userspace tweaks for certain 
architectures.
This is especially important when there's need to use a include/linux 
header which itself needs an include/asm header as in this case with the 
  the little tweak always the right asm header is included.
-- 
Andreas Steinmetz                       SPAMmers use robotrap@domdv.de

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27 17:16       ` David Woodhouse
@ 2004-11-27 22:53         ` Arnd Bergmann
  2004-11-27 23:12           ` David Woodhouse
                             ` (3 more replies)
  0 siblings, 4 replies; 185+ messages in thread
From: Arnd Bergmann @ 2004-11-27 22:53 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Randy.Dunlap, Matthew Wilcox, Tonnerre, David Howells, torvalds,
	hch, aoliva, linux-kernel, libc-hacker


[-- Attachment #1: Type: text/plain, Size: 952 bytes --]

On Sünnavend 27 November 2004 18:16, you wrote:
> On Sat, 2004-11-27 at 08:47 -0800, Randy.Dunlap wrote:
> > Speaking of more explicit, there are various asm-ARCH header
> > files that do or do not hide (via __KERNEL__) interfaces
> > such as:      get_unaligned()
> > and the atomic operations.
> > 
> > So are these Linux kernel exported APIs, or do they belong
> > in some library?
> 
> Both of those are kernel-private and should not be visible.

The problem with these (atomic.h, bitops.h, byteorder.h, div64.h,
list.h, spinlock.h, unaligned.h and xor.h) is that they provide
functionality that is needed by many user application but not
provided by the compiler or libc. 

While I agree that it is an absolutely evil concept to include
them from the kernel headers, we have to face that by not installing
them, lots of this existing evil user code will be broken even
more and someone has to pick up the pieces.

	Arnd <><

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27 22:53         ` Arnd Bergmann
@ 2004-11-27 23:12           ` David Woodhouse
  2004-11-27 23:27           ` Randy.Dunlap
                             ` (2 subsequent siblings)
  3 siblings, 0 replies; 185+ messages in thread
From: David Woodhouse @ 2004-11-27 23:12 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Randy.Dunlap, Matthew Wilcox, Tonnerre, dhowells, torvalds, hch,
	aoliva, linux-kernel, libc-hacker

On Sat, 2004-11-27 at 23:53 +0100, Arnd Bergmann wrote:
> The problem with these (atomic.h, bitops.h, byteorder.h, div64.h,
> list.h, spinlock.h, unaligned.h and xor.h) is that they provide
> functionality that is needed by many user application but not
> provided by the compiler or libc. 
> 
> While I agree that it is an absolutely evil concept to include
> them from the kernel headers, we have to face that by not installing
> them, lots of this existing evil user code will be broken even
> more and someone has to pick up the pieces.

The existing evil user code is broken already. Any of the above can be
implemented in a such a way that it only works in the kernel anyway.

We don't need to preserve it any more than we need to preserve evil
broken code which defines __KERNEL__ before including certain headers.
We're supposed to be cleaning this crap up. If we want every evil broken
hack that's currently possible to _remain_ possible, it's never going to
work.
 
-- 
dwmw2


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27 21:49       ` Andreas Steinmetz
@ 2004-11-27 23:21         ` Arnd Bergmann
  2004-11-28 13:24           ` Andreas Steinmetz
  0 siblings, 1 reply; 185+ messages in thread
From: Arnd Bergmann @ 2004-11-27 23:21 UTC (permalink / raw)
  To: Andreas Steinmetz
  Cc: Sam Ravnborg, David Howells, torvalds, hch, matthew, dwmw2,
	aoliva, linux-kernel, libc-hacker


[-- Attachment #1: Type: text/plain, Size: 1770 bytes --]

On Sünnavend 27 November 2004 22:49, Andreas Steinmetz wrote:
> Ok, this is not in the kernel, but think of the following: an 
> application needs and include/asm header. The application may be 
> compiled as 32 bit and as 64 bit. Then you could use a real asm 
> directory instead of a symlink and use, e.g. for asm/byteorder.h:
> 
> #ifdef __x86_64__
> #include <asm-x86_64/byteorder.h>
> #else
> #include <asm-i386/byteorder.h>
> #endif
> 

I think we can get rid of this hack when we move to split kernel headers.
parisc, s390 and mips already have combined headers, and it should not be
too hard to combine the user ABI headers for sparc, ppc and x86_64 as well,
without having to merge the complete architecture and kernel header trees
for them.

Building for ppc64 would then use something like 'gcc -Iinclude
-Iinclude/ppc64 -Iinclude/user -Iinclude/user/ppc/' for a layout
that has files in

include/linux/foo.h
include/$arch/asm/foo.h
include/generic/asm/foo.h
include/user/linux/foo.h           (installed as <linux/foo.h>)
include/user/$arch32/asm/foo.h     (installed as <asm/foo.h>
include/user/asm-generic/foo.h     (installed as <asm-generic/foo.h>

[Note how the first three become optional in this scheme. For files
 that need an internal version an a user version, ppc64/asm/foo.h
 can use #include_next <asm/foo.h> to get to generic/asm/foo.h and
 user/ppc/asm/foo.h.]

For ia64, the headers should probably stay separate because I would expect
them to differ more from i386/x86_64 than the other dual 32/64 architectures.
I don't think that is a problem because you need separate tool chains for
ia64 and i386 anyway, while the others can share the libc headers and the
compiler backend already.

	Arnd <><

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27 22:53         ` Arnd Bergmann
  2004-11-27 23:12           ` David Woodhouse
@ 2004-11-27 23:27           ` Randy.Dunlap
  2004-11-27 23:32             ` David Woodhouse
  2004-11-28  7:13           ` Arjan van de Ven
  2004-11-28  7:20           ` Christoph Hellwig
  3 siblings, 1 reply; 185+ messages in thread
From: Randy.Dunlap @ 2004-11-27 23:27 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: David Woodhouse, Matthew Wilcox, Tonnerre, David Howells,
	torvalds, hch, aoliva, linux-kernel, libc-hacker

Arnd Bergmann wrote:
> On Sünnavend 27 November 2004 18:16, you wrote:
> 
>>On Sat, 2004-11-27 at 08:47 -0800, Randy.Dunlap wrote:
>>
>>>Speaking of more explicit, there are various asm-ARCH header
>>>files that do or do not hide (via __KERNEL__) interfaces
>>>such as:      get_unaligned()
>>>and the atomic operations.
>>>
>>>So are these Linux kernel exported APIs, or do they belong
>>>in some library?
>>
>>Both of those are kernel-private and should not be visible.
> 
> 
> The problem with these (atomic.h, bitops.h, byteorder.h, div64.h,
> list.h, spinlock.h, unaligned.h and xor.h) is that they provide
> functionality that is needed by many user application but not
> provided by the compiler or libc. 
> 
> While I agree that it is an absolutely evil concept to include
> them from the kernel headers, we have to face that by not installing
> them, lots of this existing evil user code will be broken even
> more and someone has to pick up the pieces.

That's addressing a different problem.  I agree with
David W. that we need to clean the kernel headers up.
Let libc or libxyz provide the missing functionality.
The borken programs were stealing something that wasn't
promised to them AFAIK.

IOW, let's decide the right thing to do and implement it.

-- 
~Randy

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27 23:27           ` Randy.Dunlap
@ 2004-11-27 23:32             ` David Woodhouse
  2004-11-27 23:56               ` Arnd Bergmann
  0 siblings, 1 reply; 185+ messages in thread
From: David Woodhouse @ 2004-11-27 23:32 UTC (permalink / raw)
  To: Randy.Dunlap
  Cc: Arnd Bergmann, Matthew Wilcox, Tonnerre, dhowells, torvalds, hch,
	aoliva, linux-kernel, libc-hacker

On Sat, 2004-11-27 at 15:27 -0800, Randy.Dunlap wrote:
> That's addressing a different problem.  I agree with
> David W. that we need to clean the kernel headers up.
> Let libc or libxyz provide the missing functionality.
> The borken programs were stealing something that wasn't
> promised to them AFAIK.

Not only wasn't it promised; it wasn't even working on some
architectures anyway.

-- 
dwmw2


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27 23:32             ` David Woodhouse
@ 2004-11-27 23:56               ` Arnd Bergmann
  0 siblings, 0 replies; 185+ messages in thread
From: Arnd Bergmann @ 2004-11-27 23:56 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Randy.Dunlap, Matthew Wilcox, Tonnerre, dhowells, torvalds, hch,
	aoliva, linux-kernel, libc-hacker


[-- Attachment #1: Type: text/plain, Size: 840 bytes --]

On Sünndag 28 November 2004 00:32, David Woodhouse wrote:
> On Sat, 2004-11-27 at 15:27 -0800, Randy.Dunlap wrote:
> > That's addressing a different problem.  I agree with
> > David W. that we need to clean the kernel headers up.
> > Let libc or libxyz provide the missing functionality.
> > The borken programs were stealing something that wasn't
> > promised to them AFAIK.
> 
> Not only wasn't it promised; it wasn't even working on some
> architectures anyway.
> 
Well, at least on s390 some effort was spent on making it work
(see e.g. asm-s390/spinlock.h). I don't really mind removing
that functionality, but I'd prefer still installing some files
into /usr/include/asm/spinlock.h et.al that contain an #error
and a pointer to an explanation, so users can complain to the
right people (i.e. not me ;-).

	Arnd <><

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27 22:53         ` Arnd Bergmann
  2004-11-27 23:12           ` David Woodhouse
  2004-11-27 23:27           ` Randy.Dunlap
@ 2004-11-28  7:13           ` Arjan van de Ven
  2004-11-28 12:03             ` Arnd Bergmann
  2004-11-28 16:21             ` Jakub Jelinek
  2004-11-28  7:20           ` Christoph Hellwig
  3 siblings, 2 replies; 185+ messages in thread
From: Arjan van de Ven @ 2004-11-28  7:13 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: David Woodhouse, Randy.Dunlap, Matthew Wilcox, Tonnerre,
	David Howells, torvalds, hch, aoliva, linux-kernel, libc-hacker


> The problem with these (atomic.h

that is a very non portable header and there are several good
alternatives (see the apr library for example). In fact. atomic.h is
*dangerous* in userspace, it is only atomic if CONFIG_SMP is set, so if
you compile your app on a machine without that set and then run it on an
smp machine, you are not atomic.

>
> , bitops.h

again not portable 

>
> , byteorder.h, 

there are perfectly good alternatives in glibc

>
> div64.h,

huh? what is wrong with "/" in C

> list.h

this one I can see

>
> , spinlock.h

EHHHH????? Spinlocks in userland? You got to be kidding.


> , unaligned.h 

weird

> and xor.h) 

xor.h is very raid specific (and GPL with lots of code, so a license
trap)



^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27 22:53         ` Arnd Bergmann
                             ` (2 preceding siblings ...)
  2004-11-28  7:13           ` Arjan van de Ven
@ 2004-11-28  7:20           ` Christoph Hellwig
  3 siblings, 0 replies; 185+ messages in thread
From: Christoph Hellwig @ 2004-11-28  7:20 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: David Woodhouse, Randy.Dunlap, Matthew Wilcox, Tonnerre,
	David Howells, torvalds, hch, aoliva, linux-kernel, libc-hacker

> The problem with these (atomic.h, bitops.h, byteorder.h, div64.h,
> list.h, spinlock.h, unaligned.h and xor.h) is that they provide
> functionality that is needed by many user application but not
> provided by the compiler or libc. 

It's already broken on many architectures and just happens to work
on some.


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-28  7:13           ` Arjan van de Ven
@ 2004-11-28 12:03             ` Arnd Bergmann
  2004-11-28 12:19               ` Arjan van de Ven
  2004-11-28 12:28               ` Wichert Akkerman
  2004-11-28 16:21             ` Jakub Jelinek
  1 sibling, 2 replies; 185+ messages in thread
From: Arnd Bergmann @ 2004-11-28 12:03 UTC (permalink / raw)
  To: Arjan van de Ven
  Cc: David Woodhouse, Randy.Dunlap, Matthew Wilcox, Tonnerre,
	David Howells, torvalds, hch, aoliva, linux-kernel, libc-hacker


[-- Attachment #1: Type: text/plain, Size: 2215 bytes --]

Ok, I've looked for places where someone actually tried using
the kernel headers by googling for /usr/include/asm/foo.h.
The good news is that marking these files broken in 
glibc-kernheaders has already pointed most authors to the
source of the problem.

On Sünndag 28 November 2004 08:13, Arjan van de Ven wrote:
> 
> > The problem with these (atomic.h
> 
> that is a very non portable header and there are several good
> alternatives (see the apr library for example). In fact. atomic.h is
> *dangerous* in userspace, it is only atomic if CONFIG_SMP is set, so if
> you compile your app on a machine without that set and then run it on an
> smp machine, you are not atomic.

Yes, it appears that most people who attempted to use this
have already been bitten so hard that they stopped trying.

> >
> > , bitops.h
> 
> again not portable 

netbase-ping6 tried to use it, along with some other applications that
I had not heard of.

> >
> > , byteorder.h, 
> 
> there are perfectly good alternatives in glibc

Google found abuses of byteorder.h in kdeedu, dbootstrap and netatalk,
I would expect to find many more if I kept looking.

> >
> > div64.h,
> 
> huh? what is wrong with "/" in C

Ok, probably nobody has tried to do funny things with this one.

> > list.h
> 
> this one I can see

Surprisingly few abuses. It is already been protected with
#ifdef __KERNEL__ for some time.

> >
> > , spinlock.h
> 
> EHHHH????? Spinlocks in userland? You got to be kidding.

I don't think it's that uncommon to use spinlocks in user
space, IIRC samba (tdb) and some databases implement their
own version of user spinlocks. The kernel implementation
is used at least in "Open Runtime Platform" and in "Chrony".
Chrony even has an FAQ entry on how to circumvent the 
"#ifndef __KERNEL__" that was added in Red Hat Linux...

> > , unaligned.h 
> 
> weird

At least one application that I use every day (vdr) tried including
asm/unaligned.h, but it's rather uncommon otherwise.

> > and xor.h) 
> 
> xor.h is very raid specific (and GPL with lots of code, so a license
> trap)

Right, this one has never been used from user space AFAICS.

	Arnd <><

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-28 12:03             ` Arnd Bergmann
@ 2004-11-28 12:19               ` Arjan van de Ven
  2004-11-28 16:34                 ` Kevin Puetz
  2004-11-28 12:28               ` Wichert Akkerman
  1 sibling, 1 reply; 185+ messages in thread
From: Arjan van de Ven @ 2004-11-28 12:19 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: David Woodhouse, Randy.Dunlap, Matthew Wilcox, Tonnerre,
	David Howells, torvalds, hch, aoliva, linux-kernel, libc-hacker


> > The problem with these (atomic.h
> > 
> > that is a very non portable header and there are several good
> > alternatives (see the apr library for example). In fact. atomic.h is
> > *dangerous* in userspace, it is only atomic if CONFIG_SMP is set, so if
> > you compile your app on a machine without that set and then run it on an
> > smp machine, you are not atomic.
> 
> Yes, it appears that most people who attempted to use this
> have already been bitten so hard that they stopped trying.

and for people who'd be tempted for new code it should at least warn if
not error even right now


> > > , byteorder.h, 
> > 
> > there are perfectly good alternatives in glibc
> 
> Google found abuses of byteorder.h in kdeedu, dbootstrap and netatalk,
> I would expect to find many more if I kept looking.

it's still very much abuse, and quite nasty for dual endian platforms ;)

> > > , spinlock.h
> > 
> > EHHHH????? Spinlocks in userland? You got to be kidding.
> 
> I don't think it's that uncommon to use spinlocks in user
> space, IIRC samba (tdb) and some databases implement their
> own version of user spinlocks.

implementing your own is still evil, but glibc provides similar
constructs already. busy waiting in userspace is evil anyway (use futex
:) and.... just as with atomic.h, spinlock.h only gets compiled in for
CONFIG_SMP so the same caveats apply

>  The kernel implementation
> is used at least in "Open Runtime Platform" and in "Chrony".
> Chrony even has an FAQ entry on how to circumvent the 
> "#ifndef __KERNEL__" that was added in Red Hat Linux...

Chrony then needs to make their own (by copying and cleaning up for all
I care).




^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-28 12:03             ` Arnd Bergmann
  2004-11-28 12:19               ` Arjan van de Ven
@ 2004-11-28 12:28               ` Wichert Akkerman
  2004-11-28 12:44                 ` Arjan van de Ven
  1 sibling, 1 reply; 185+ messages in thread
From: Wichert Akkerman @ 2004-11-28 12:28 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Arjan van de Ven, David Woodhouse, Randy.Dunlap, Matthew Wilcox,
	Tonnerre, David Howells, torvalds, hch, aoliva, linux-kernel,
	libc-hacker

Previously Arnd Bergmann wrote:
> Ok, I've looked for places where someone actually tried using
> the kernel headers by googling for /usr/include/asm/foo.h.
> The good news is that marking these files broken in 
> glibc-kernheaders has already pointed most authors to the
> source of the problem.

If you want a challenge try if you can get strace to compile with
glibc-kernheaders. 

Wichert.

-- 
Wichert Akkerman <wichert@wiggy.net>    It is simple to make things.
http://www.wiggy.net/                   It is hard to make things simple.

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-28 12:28               ` Wichert Akkerman
@ 2004-11-28 12:44                 ` Arjan van de Ven
  0 siblings, 0 replies; 185+ messages in thread
From: Arjan van de Ven @ 2004-11-28 12:44 UTC (permalink / raw)
  To: Wichert Akkerman
  Cc: Arnd Bergmann, David Woodhouse, Randy.Dunlap, Matthew Wilcox,
	Tonnerre, David Howells, torvalds, hch, aoliva, linux-kernel,
	libc-hacker

On Sun, 2004-11-28 at 13:28 +0100, Wichert Akkerman wrote:
> Previously Arnd Bergmann wrote:
> > Ok, I've looked for places where someone actually tried using
> > the kernel headers by googling for /usr/include/asm/foo.h.
> > The good news is that marking these files broken in 
> > glibc-kernheaders has already pointed most authors to the
> > source of the problem.
> 
> If you want a challenge try if you can get strace to compile with
> glibc-kernheaders. 

the rh strace package seems to cope....

and while glibc-kernheaders hides the kernel internal structures, it
does expose all the external ones...




^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27 23:21         ` Arnd Bergmann
@ 2004-11-28 13:24           ` Andreas Steinmetz
  2004-11-28 13:40             ` Arnd Bergmann
  0 siblings, 1 reply; 185+ messages in thread
From: Andreas Steinmetz @ 2004-11-28 13:24 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Sam Ravnborg, David Howells, torvalds, hch, matthew, dwmw2,
	aoliva, linux-kernel, libc-hacker

Arnd Bergmann wrote:
>>#ifdef __x86_64__
>>#include <asm-x86_64/byteorder.h>
>>#else
>>#include <asm-i386/byteorder.h>
>>#endif
>>
> 
> 
> I think we can get rid of this hack when we move to split kernel headers.
> parisc, s390 and mips already have combined headers, and it should not be
> too hard to combine the user ABI headers for sparc, ppc and x86_64 as well,
> without having to merge the complete architecture and kernel header trees
> for them.
> 

If you can produce merged x86/x86_64 ABI headers this would be a nice 
solution.
-- 
Andreas Steinmetz                       SPAMmers use robotrap@domdv.de

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-28 13:24           ` Andreas Steinmetz
@ 2004-11-28 13:40             ` Arnd Bergmann
  0 siblings, 0 replies; 185+ messages in thread
From: Arnd Bergmann @ 2004-11-28 13:40 UTC (permalink / raw)
  To: Andreas Steinmetz
  Cc: Sam Ravnborg, David Howells, torvalds, hch, matthew, dwmw2,
	aoliva, linux-kernel, libc-hacker, ak


[-- Attachment #1: Type: text/plain, Size: 686 bytes --]

On Sünndag 28 November 2004 14:24, Andreas Steinmetz wrote:
> > I think we can get rid of this hack when we move to split kernel headers.
> > parisc, s390 and mips already have combined headers, and it should not be
> > too hard to combine the user ABI headers for sparc, ppc and x86_64 as well,
> > without having to merge the complete architecture and kernel header trees
> > for them.
> > 
> 
> If you can produce merged x86/x86_64 ABI headers this would be a nice 
> solution.

I can do that (and the same for ppc/ppc64) as soon as we have
a) A patch that first splits out the ABI headers for i386
b) persuaded ak that he will look at what I come up with

	Arnd <><

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-28  7:13           ` Arjan van de Ven
  2004-11-28 12:03             ` Arnd Bergmann
@ 2004-11-28 16:21             ` Jakub Jelinek
  1 sibling, 0 replies; 185+ messages in thread
From: Jakub Jelinek @ 2004-11-28 16:21 UTC (permalink / raw)
  To: Arjan van de Ven
  Cc: Arnd Bergmann, David Woodhouse, Randy.Dunlap, Matthew Wilcox,
	Tonnerre, David Howells, torvalds, hch, aoliva, linux-kernel,
	libc-hacker

On Sun, Nov 28, 2004 at 08:13:39AM +0100, Arjan van de Ven wrote:
> > , spinlock.h
> 
> EHHHH????? Spinlocks in userland? You got to be kidding.

There is pthread_spin_{init,lock,unlock,trylock,destroy}
in libpthread.so.

	Jakub

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-28 12:19               ` Arjan van de Ven
@ 2004-11-28 16:34                 ` Kevin Puetz
  0 siblings, 0 replies; 185+ messages in thread
From: Kevin Puetz @ 2004-11-28 16:34 UTC (permalink / raw)
  To: linux-kernel

Arjan van de Ven wrote:

> implementing your own is still evil, but glibc provides similar
> constructs already. busy waiting in userspace is evil anyway (use futex
> :) and.... just as with atomic.h, spinlock.h only gets compiled in for
> CONFIG_SMP so the same caveats apply

Yeah, this is on it's way out now that futexes exist, but KDE (for one) used
to use spinlocks for a thread-pooled malloc (spin-sleep-spin-sleep though,
calling either sched_yield or nanosleep if the lock-grab failed - so it
wasn't a pure busy-wait). It didn't reuse spinlock.h from the kernel, but
it did use the general concept, since the old pre-futex posix mutexes were
crazy-slow for a lock which was heavily used but rarely contended.


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27  3:24     ` Greg KH
@ 2004-11-28 23:37       ` Paul Mackerras
  2004-11-29  1:28         ` Linus Torvalds
  0 siblings, 1 reply; 185+ messages in thread
From: Paul Mackerras @ 2004-11-28 23:37 UTC (permalink / raw)
  To: Greg KH
  Cc: David Woodhouse, Matthew Wilcox, David Howells, torvalds, hch,
	aoliva, linux-kernel, libc-hacker

Greg KH writes:

> The justification is that it doesn't properly describe the variable size
> (think userspace 32 bit variable vs. kernelspace 32 bit variable.)  We
> need to stick with the proper __u32 type variables for data that crosses
> the userspace/kernelspace boundry.

uint32_t is defined to be exactly 32 bits wide, so where's the problem
in using it instead of __u32 in the headers that describe the
user/kernel interface?  (Ditto for uint{8,16,64}_t, of course.)

Paul.

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-28 23:37       ` Paul Mackerras
@ 2004-11-29  1:28         ` Linus Torvalds
  2004-11-29  4:36           ` Jeff Garzik
                             ` (5 more replies)
  0 siblings, 6 replies; 185+ messages in thread
From: Linus Torvalds @ 2004-11-29  1:28 UTC (permalink / raw)
  To: Paul Mackerras
  Cc: Greg KH, David Woodhouse, Matthew Wilcox, David Howells, hch,
	aoliva, linux-kernel, libc-hacker



On Mon, 29 Nov 2004, Paul Mackerras wrote:
> 
> uint32_t is defined to be exactly 32 bits wide, so where's the problem
> in using it instead of __u32 in the headers that describe the
> user/kernel interface?  (Ditto for uint{8,16,64}_t, of course.

Ok, this discussion has gone on for too long anyway, but let's make it
easier for everybody. The kernel uses u8/u16/u32 because:

	- the kernel should not depend on, or pollute user-space naming.
	  YOU MUST NOT USE "uint32_t" when that may not be defined, and
	  user-space rules for when it is defined are arcane and totally
	  arbitrary.

	- since the kernel cannot use those types for anything that is 
	  visible to user space anyway, there has to be alternate names. 
	  The tradition is to prepend two underscores, so the kernel would 
	  have to use "__uint32_t" etc for its header files.

	- at that point, there's no longer any valid argument that it's a 
	  "standard type" (it ain't), and I personally find it a lot more 
	  readable to just use the types that the kernel has always used:  
	  __u8/__u16/__u32. For stuff that is only used for the kernel, 
	  the shorter "u8/u16/u32" versions may be used.

In short: having the kernel use the same names as user space is ACTIVELY 
BAD, exactly because those names have standards-defined visibility, which 
means that the kernel _cannot_ use them in all places anyway. So don't 
even _try_. 

On the bigger question of what to do with kernel headers in general, let's 
just make one thing clear: the kernel headers are for the kernel, and big 
and painful re-organizations that don't help _existing_ user space are not 
going to happen.

In particular, any re-organization that breaks _existing_ uses is totally
pointless. If you break existing uses, you might as well _not_ re-
organize, since if you consider kernel headers to be purely kernel-
internal (like they should be, but hey, reality trumps any wishes we might 
have), then the current organization is perfectly fine.

So I think this whole discussion has been largely pointless. We undeniably
have existing users of kernel headers. That's just a fact.  If we break
them, it doesn't _matter_ how the kernel headers look, and then "existing
practice" is about as good an organization as anything, and breaking
things just to break things is not something I'm in the least interested
in. "Beauty"  comes secondary to "usefulness".

So I would encourage people to think about ways to clean up some of the 
worst warts, but take into account:
 - testing it out with whatever random collection of old distributions and 
   special applications is almost impossible. So every single step of the 
   way should be (a) small and (b) obviously not break any current users.
 - cleanup just for the sake of cleanup always needs to take pain into 
   account. If you cannot make each small step "worth it", then just don't 
   do it. If the "cleanup" just adds another file and doesn't actually 
   _help_ anything that you can point to, it's not a cleanup, it's just an 
   exercise in wanking.

IOW, I seriously doubt any "let's reorganize header files just to make it
look good" _ever_ accomplished anything. But if there are _specific_
header files that have _specific_ problems, let's look at maybe solving
those. And if you cannot point to a specific problem with a suggested
specific solution, please don't cc me.

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29  1:28         ` Linus Torvalds
@ 2004-11-29  4:36           ` Jeff Garzik
  2004-11-29  4:57             ` Al Viro
                               ` (2 more replies)
  2004-11-29  9:42           ` David Woodhouse
                             ` (4 subsequent siblings)
  5 siblings, 3 replies; 185+ messages in thread
From: Jeff Garzik @ 2004-11-29  4:36 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox,
	David Howells, hch, aoliva, linux-kernel, libc-hacker,
	Mariusz Mazur, Arjan van de Ven

Linus Torvalds wrote:
> In short: having the kernel use the same names as user space is ACTIVELY 
> BAD, exactly because those names have standards-defined visibility, which 
> means that the kernel _cannot_ use them in all places anyway. So don't 
> even _try_. 
> 
> On the bigger question of what to do with kernel headers in general, let's 
> just make one thing clear: the kernel headers are for the kernel, and big 
> and painful re-organizations that don't help _existing_ user space are not 
> going to happen.
> 
> In particular, any re-organization that breaks _existing_ uses is totally
> pointless. If you break existing uses, you might as well _not_ re-
> organize, since if you consider kernel headers to be purely kernel-
> internal (like they should be, but hey, reality trumps any wishes we might 
> have), then the current organization is perfectly fine.


I don't think any drastic reorganization is even necessary.

Mariusz Mazur <mmazur@kernel.pl> updates 
http://ep09.pld-linux.org/~mmazur/linux-libc-headers/ for each 2.6.x 
kernel release.  linux-libc-headers are the kernel headers, with all the 
kernel-specific stuff stripped out.  i.e. userland ABI only.  Not sure 
how many distros have started picking that up yet...  I think Arjan said 
Fedora Core had, or would.

If people want to go beyond that, IMHO it would be simple and easy to 
start putting new kernel headers in include/kernel (or somesuch).  That 
way there are no massive reorganizations; kernel-specific stuff gets 
slowly migrated to a kernel-specific area.

	Jeff



^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29  4:36           ` Jeff Garzik
@ 2004-11-29  4:57             ` Al Viro
  2004-11-29  5:27               ` Tim Hockin
  2004-11-29  7:51             ` Arjan van de Ven
  2004-12-05  0:49             ` Rob Landley
  2 siblings, 1 reply; 185+ messages in thread
From: Al Viro @ 2004-11-29  4:57 UTC (permalink / raw)
  To: Jeff Garzik
  Cc: Linus Torvalds, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, David Howells, hch, aoliva, linux-kernel,
	libc-hacker, Mariusz Mazur, Arjan van de Ven

On Sun, Nov 28, 2004 at 11:36:22PM -0500, Jeff Garzik wrote:
> If people want to go beyond that, IMHO it would be simple and easy to 
> start putting new kernel headers in include/kernel (or somesuch).  That 
> way there are no massive reorganizations; kernel-specific stuff gets 
> slowly migrated to a kernel-specific area.

ITYM "to areas where it actually gets used".  A _lot_ in include/* is
used only by a couple of drivers and should've been sitting in
drivers/*/* instead.

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29  4:57             ` Al Viro
@ 2004-11-29  5:27               ` Tim Hockin
  0 siblings, 0 replies; 185+ messages in thread
From: Tim Hockin @ 2004-11-29  5:27 UTC (permalink / raw)
  To: Al Viro
  Cc: Jeff Garzik, Linus Torvalds, Paul Mackerras, Greg KH,
	David Woodhouse, Matthew Wilcox, David Howells, hch, aoliva,
	linux-kernel, libc-hacker, Mariusz Mazur, Arjan van de Ven

On Mon, Nov 29, 2004 at 04:57:06AM +0000, Al Viro wrote:
> ITYM "to areas where it actually gets used".  A _lot_ in include/* is
> used only by a couple of drivers and should've been sitting in
> drivers/*/* instead.

Amen to that!  Just because it is a header doesn't mean it belongs in
include.  Just because it is a #define or a struct definition does not
mean it belongs in a header.

So many people don't get the idea of exposed vs private interfaces.

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29  4:36           ` Jeff Garzik
  2004-11-29  4:57             ` Al Viro
@ 2004-11-29  7:51             ` Arjan van de Ven
  2004-12-05  0:49             ` Rob Landley
  2 siblings, 0 replies; 185+ messages in thread
From: Arjan van de Ven @ 2004-11-29  7:51 UTC (permalink / raw)
  To: Jeff Garzik
  Cc: Linus Torvalds, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, David Howells, hch, aoliva, linux-kernel,
	libc-hacker, Mariusz Mazur


On Sun, Nov 28, 2004 at 11:36:22PM -0500, Jeff Garzik wrote:
> >In particular, any re-organization that breaks _existing_ uses is totally
> >pointless. If you break existing uses, you might as well _not_ re-
> >organize, since if you consider kernel headers to be purely kernel-
> >internal (like they should be, but hey, reality trumps any wishes we might 
> >have), then the current organization is perfectly fine.
> 
> 
> I don't think any drastic reorganization is even necessary.

Well, we want things to be split. However a split doesn't mean a
reorganisation as far as userspace visibility is concerned; the filenames
can still be the same, and the typenames etc etc, you just do the kernel
internal additions in a different dir/header

Not breaking userland ever is a dream only, anytime we touch any header,
some userland breaks (they use our spinlock code even!)

> kernel-specific stuff stripped out.  i.e. userland ABI only.  Not sure 
> how many distros have started picking that up yet...  I think Arjan said 
> Fedora Core had, or would.

nope.

Fedora has it's own set of cleaned up headers. Cleaned up in the sense that
tehre's no #ifdef KERNEL anymore, no inlines (because that's a license trap,
and in addition, all inlines were kernel specific anyway) and all structs
which had spinlocks / semaphores and other kernel private structures in are
removed as well (because they are clearly kernel internal).

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29  1:28         ` Linus Torvalds
  2004-11-29  4:36           ` Jeff Garzik
@ 2004-11-29  9:42           ` David Woodhouse
  2004-11-30  8:08             ` Alex Riesen
  2004-12-01 11:46             ` Ralf Baechle
  2004-11-29  9:53           ` Paul Mackerras
                             ` (3 subsequent siblings)
  5 siblings, 2 replies; 185+ messages in thread
From: David Woodhouse @ 2004-11-29  9:42 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Paul Mackerras, Greg KH, Matthew Wilcox, David Howells, hch,
	aoliva, linux-kernel, libc-hacker

On Sun, 2004-11-28 at 17:28 -0800, Linus Torvalds wrote:
> In particular, any re-organization that breaks _existing_ uses is totally
> pointless. If you break existing uses, you might as well _not_ re-
> organize, since if you consider kernel headers to be purely kernel-
> internal (like they should be, but hey, reality trumps any wishes we might 
> have), then the current organization is perfectly fine.

Agreed, with the proviso that the inclusion of bitops.h, spinlock.h, and
other kernel-private stuff like that should not be considered 'existing
uses'. It's _already_ broken. It won't compile on some architectures,
and will silently do the wrong thing on others. Causing the compile to
fail consistently would be a _feature_.

> So I think this whole discussion has been largely pointless. We undeniably
> have existing users of kernel headers. That's just a fact.  If we break
> them, it doesn't _matter_ how the kernel headers look, and then "existing
> practice" is about as good an organization as anything, and breaking
> things just to break things is not something I'm in the least interested
> in. "Beauty"  comes secondary to "usefulness".

Usefulness is what we're after. Preventing the naïve user from including
spinlock.h and thinking that it'll give useful spinlocks would be
useful.

I've lost track of the number of times things have broken because of
incorrect use of kernel headers from userspace. That's what we're trying
to fix -- by putting only the bits which are _supposed_ to be visible
into files which userspace sees, where we know they define part of the
userspace API and hence we can be extremely careful when editing them. 

I don't think it makes sense at this point for us to bury our collective
heads in the sand and pretend there isn't a problem here that's worth
fixing.

I agree that it should be obviously correct though -- and that's why
we're trying to end up with a structure that in the first pass would
give us in userspace essentially what we already have in the various
glibc-kernheaders packages, but without the constant and unnecessary
need for some poor sod to keep those up to date by hand.

-- 
dwmw2



^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29  1:28         ` Linus Torvalds
  2004-11-29  4:36           ` Jeff Garzik
  2004-11-29  9:42           ` David Woodhouse
@ 2004-11-29  9:53           ` Paul Mackerras
  2004-11-29  9:57             ` Arjan van de Ven
  2004-11-29 11:41           ` Mariusz Mazur
                             ` (2 subsequent siblings)
  5 siblings, 1 reply; 185+ messages in thread
From: Paul Mackerras @ 2004-11-29  9:53 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Greg KH, David Woodhouse, Matthew Wilcox, David Howells, hch,
	aoliva, linux-kernel, libc-hacker

Linus Torvalds writes:

> In particular, any re-organization that breaks _existing_ uses is totally
> pointless. If you break existing uses, you might as well _not_ re-
> organize, since if you consider kernel headers to be purely kernel-
> internal (like they should be, but hey, reality trumps any wishes we might 
> have), then the current organization is perfectly fine.

Recently I had some users complaining because their userspace program
that includes <asm/atomic.h> compiles OK as a ppc64 program but not as
a ppc32 program.  The reason for that is that asm-ppc/atomic.h has
#ifdef __KERNEL__ around the inline definitions of atomic_inc et al.,
but asm-ppc64/atomic.h doesn't.

My response was that they shouldn't be using <asm/atomic.h> because
(a) it's an internal kernel header, not part of the kernel ABI, (b)
it's not portable (it happens to work on some popular architectures,
but won't work on all) and (c) they're really only entitled to use
those particular inline function definitions in GPL'd programs anyway.

To which the obvious reply is "well then why is it in /usr/include?"
It's because the glibc build process copies all the kernel headers to
/usr/include, but that's not an answer from a user's point of view.

I'd be interested to hear your take on this.  Should we try to make
our atomics easy and safe for userspace to use (including putting them
under the LGPL)?  Or can you see a better solution?

Paul.

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29  9:53           ` Paul Mackerras
@ 2004-11-29  9:57             ` Arjan van de Ven
  2004-11-29 10:01               ` David Woodhouse
  0 siblings, 1 reply; 185+ messages in thread
From: Arjan van de Ven @ 2004-11-29  9:57 UTC (permalink / raw)
  To: Paul Mackerras
  Cc: Linus Torvalds, Greg KH, David Woodhouse, Matthew Wilcox,
	David Howells, hch, aoliva, linux-kernel, libc-hacker

On Mon, 2004-11-29 at 20:53 +1100, Paul Mackerras wrote:
> Linus Torvalds writes:
> 
> > In particular, any re-organization that breaks _existing_ uses is totally
> > pointless. If you break existing uses, you might as well _not_ re-
> > organize, since if you consider kernel headers to be purely kernel-
> > internal (like they should be, but hey, reality trumps any wishes we might 
> > have), then the current organization is perfectly fine.
> 
> Recently I had some users complaining because their userspace program
> that includes <asm/atomic.h> compiles OK as a ppc64 program but not as
> a ppc32 program.  The reason for that is that asm-ppc/atomic.h has
> #ifdef __KERNEL__ around the inline definitions of atomic_inc et al.,
> but asm-ppc64/atomic.h doesn't.
> 
> My response was that they shouldn't be using <asm/atomic.h> because
> (a) it's an internal kernel header, not part of the kernel ABI, (b)
> it's not portable (it happens to work on some popular architectures,
> but won't work on all) and (c) they're really only entitled to use
> those particular inline function definitions in GPL'd programs anyway.

and (d) on x86 at least the "atomic" part of the atomic ops are #ifdef
CONFIG_SMP, which might well not be set in userland... so you get a
false sense of atomicity.

> I'd be interested to hear your take on this.  Should we try to make
> our atomics easy and safe for userspace to use (including putting them
> under the LGPL)?  Or can you see a better solution?

it's not the kernel's job to provide this functionality imo. 
glibc could. glib does. apr does. there's several options already.
apr is BSD licensed for exanpmle, glib LGPL.



^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29  9:57             ` Arjan van de Ven
@ 2004-11-29 10:01               ` David Woodhouse
  0 siblings, 0 replies; 185+ messages in thread
From: David Woodhouse @ 2004-11-29 10:01 UTC (permalink / raw)
  To: Arjan van de Ven
  Cc: Paul Mackerras, Linus Torvalds, Greg KH, Matthew Wilcox,
	David Howells, hch, aoliva, linux-kernel, libc-hacker

On Mon, 2004-11-29 at 10:57 +0100, Arjan van de Ven wrote:
> > I'd be interested to hear your take on this.  Should we try to make
> > our atomics easy and safe for userspace to use (including putting them
> > under the LGPL)?  Or can you see a better solution?
> 
> it's not the kernel's job to provide this functionality imo. 

Sometimes the functionality would have to be entirely reimplemented to
make it viable for userspace. For FR-V we set aside a condition register
for use in the kernel, so we can emulate ll/sc with multiple-issue
instructions. That can never work in userspace.

We can't just pretend there isn't a problem here.

-- 
dwmw2



^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__ 
  2004-11-25 15:13 [RFC] Splitting kernel headers and deprecating __KERNEL__ David Howells
                   ` (8 preceding siblings ...)
  2004-11-27 21:03 ` Sam Ravnborg
@ 2004-11-29 11:08 ` David Howells
  2004-11-29 20:01 ` H. Peter Anvin
  10 siblings, 0 replies; 185+ messages in thread
From: David Howells @ 2004-11-29 11:08 UTC (permalink / raw)
  To: Grzegorz Kulewski; +Cc: torvalds, hch, matthew, dwmw2, aoliva, linux-kernel


Grzegorz Kulewski <kangur@polcom.net>:
> >     (b) Make kernel file #include the user file.
> 
> Does kernel really need to include user headers? When it is definition of 
> some const then it should be defined in one file (to be sure it has only 
> one definition). But user headers may have some compatibility hacks that 
> kernel do not need (and even maybe does not want) to have.
> 
> How you will handle that?

I must have mis-explained it. I meant:

> >     (b) Make kernel file #include the user/ file.

I don't actually mean that the kernel should in anyway touch userspace
headers. I was referring to the bits split out into user-xxxx/ directories.

David

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29  1:28         ` Linus Torvalds
                             ` (2 preceding siblings ...)
  2004-11-29  9:53           ` Paul Mackerras
@ 2004-11-29 11:41           ` Mariusz Mazur
  2004-11-29 17:09           ` Alexandre Oliva
  2004-12-14  5:51           ` Werner Almesberger
  5 siblings, 0 replies; 185+ messages in thread
From: Mariusz Mazur @ 2004-11-29 11:41 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox,
	David Howells, hch, aoliva, linux-kernel, libc-hacker,
	Jeff Garzik

On poniedziałek 29 listopad 2004 02:28, Linus Torvalds wrote:
> So I would encourage people to think about ways to clean up some of the
> worst warts, but take into account:
>  - testing it out with whatever random collection of old distributions and
>    special applications is almost impossible. So every single step of the
>    way should be (a) small and (b) obviously not break any current users.

If I understand correctly the above means "include/* userland breakage 
problems open for fixing" and the "kernel includes aren't for userland" part 
is obsolete?

>  - cleanup just for the sake of cleanup always needs to take pain into
>    account. If you cannot make each small step "worth it", then just don't
>    do it. If the "cleanup" just adds another file and doesn't actually
>    _help_ anything that you can point to, it's not a cleanup, it's just an
>    exercise in wanking.
>
> IOW, I seriously doubt any "let's reorganize header files just to make it
> look good" _ever_ accomplished anything. But if there are _specific_
> header files that have _specific_ problems, let's look at maybe solving
> those. And if you cannot point to a specific problem with a suggested
> specific solution, please don't cc me.

Potential breakage is mostly a non issue - since vanilla kernel headers are 
mostly useless for userland anyway, one must either patch them or use my 
headers (fedora's?). As for my headers - I can always provide a compatibility 
layer that will look like the old tree but in fact use the new tree.* And 
that also means application specific hacks not to break some app (I'm doing 
that now anyway, so no difference).

I'm *really* for fixing the header problem once and for all one way (lifting 
the 'kernel includes aren't for userland' ban) or the other (making a 
separate, userland available part that glibc and other apps can use cleanly). 
Maintaining a separate tree of those headers ain't much fun and it would be 
nice if I could quit doing that some time in the future.


* Fluent and (mostly) painless migration guaranteed or your money back.


-- 
In the year eighty five ten
God is gonna shake his mighty head
He'll either say,
"I'm pleased where man has been"
Or tear it down, and start again

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-26 11:58   ` David Howells
  2004-11-26 12:00     ` David Woodhouse
@ 2004-11-29 16:34     ` Alexandre Oliva
  1 sibling, 0 replies; 185+ messages in thread
From: Alexandre Oliva @ 2004-11-29 16:34 UTC (permalink / raw)
  To: David Howells
  Cc: Matthew Wilcox, torvalds, hch, dwmw2, linux-kernel, libc-alpha

On Nov 26, 2004, David Howells <dhowells@redhat.com> wrote:

> How about calling the interface headers "kapi*/" instead of "user*/". In case
> you haven't guessed, "kapi" would be short for "kernel-api".

I've seen kapi being used to reference the api exposed by the kernel
to modules, but not the abi exposed to userland.  ukabi sounds more
appropriate for the latter, although many people might wonder what's
with this United Kingdom ABI :-)

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29  1:28         ` Linus Torvalds
                             ` (3 preceding siblings ...)
  2004-11-29 11:41           ` Mariusz Mazur
@ 2004-11-29 17:09           ` Alexandre Oliva
  2004-11-29 17:41             ` Linus Torvalds
  2004-11-30 15:33             ` David Howells
  2004-12-14  5:51           ` Werner Almesberger
  5 siblings, 2 replies; 185+ messages in thread
From: Alexandre Oliva @ 2004-11-29 17:09 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox,
	David Howells, hch, linux-kernel, libc-hacker

On Nov 28, 2004, Linus Torvalds <torvalds@osdl.org> wrote:

> On the bigger question of what to do with kernel headers in general, let's 
> just make one thing clear: the kernel headers are for the kernel, and big 
> and painful re-organizations that don't help _existing_ user space are not 
> going to happen.

So, of the following two options, which one you think everybody should
pick?

- Linux gets to define the ABI between kernel and userland, and
  userland must duplicate the contents of headers in which the kernel
  defines the kernel<->userland ABI, tracking changes in them in the
  hope that nothing falls through the cracks

- we move the kernel<->userland to a separate package, where it's
  maintained such that it can be used by both kernel and userland, and
  Linux will only build when given a pointer to the location of this
  package.


I don't think you can have it both ways.  As long as the kernel wants
freedom to set the ABI, either let people use the headers where you
define it, or maintain the ABI separately and let people use them.
Forcing others to maintain duplicates of your headers will just get
most people to use your headers for purposes they weren't meant to be
used; maintaining two similar but far from identical sets of headers
in sync is a major pain, that's why few people do it, and even fewer
people do it properly.

Refusing to help in the effort to maintain them for wishes of freedom
to tinker or allergy to external dependencies haven't worked as
excuses in the past, and I think they never will.  Duplicating the
effort is just stupid, and requiring others to do so is arrogant.  Why
shouldn't we just get together and have something that works for both?

Or how about we turn the table around and define an ABI that the
kernel must implement, and then have the kernel have to keep track of
the changes in such external ABI definition, instead of the way things
(fail to) work now?  How comfortable would you feel with this
arrangement?


My usual reply when I hear this argument that the kernel headers that
define the ABI with userland are to be used by the kernel only, and
someone else is supposed to create the headers for the same purpose to
be used by userland, is that you generally won't find the headers that
expose the interface of a library in a separate package (think
libz.{a,so} and zlib.h).  They're part of the same package, built out
of the same sources, and there's a reason for that: if they get out of
sync, it won't serve its purpose.  It makes far more sense to maintain
them together.

You won't find zlib.h in packages that just want to use zlib, unless
they contain zlib in its entirety, just in case it's not
pre-installed.  I think this makes perfect sense.  Requiring libc or a
separate package used by libc to contain copies of headers that define
the API/ABI exported by the kernel to userland is just the same sort
of nonsense, with the difference that it's a way bigger set of headers
we're talking about, which renders the problem even worse.

Sure you can take the easy way out and claim you told people they
shouldn't do what they're doing when they run into problems because of
including kernel headers from userland.  But as soon as you reject
proposals to clean things up such that we would have a set of headers
that both kernel and userland will be able to use, in a way that would
have the headers still conveniently (for you) sitting in the kernel
source tree, I can't see that you could say it's not your problem with
a straight face any more.

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29 17:09           ` Alexandre Oliva
@ 2004-11-29 17:41             ` Linus Torvalds
  2004-11-29 22:44               ` Al Viro
                                 ` (2 more replies)
  2004-11-30 15:33             ` David Howells
  1 sibling, 3 replies; 185+ messages in thread
From: Linus Torvalds @ 2004-11-29 17:41 UTC (permalink / raw)
  To: Alexandre Oliva
  Cc: Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox,
	David Howells, hch, linux-kernel, libc-hacker



On Mon, 29 Nov 2004, Alexandre Oliva wrote:
>
> > On the bigger question of what to do with kernel headers in general, let's 
> > just make one thing clear: the kernel headers are for the kernel, and big 
> > and painful re-organizations that don't help _existing_ user space are not 
> > going to happen.
> 
> So, of the following two options, which one you think everybody should
> pick?

Ok, that's pretty straightforward:

> - Linux gets to define the ABI between kernel and userland, and
>   userland must duplicate the contents of headers in which the kernel
>   defines the kernel<->userland ABI, tracking changes in them in the
>   hope that nothing falls through the cracks

This is unquestionably true. The kernel obviously _does_ define the ABI, 
and userland just lives with it. At some point you have to track things, 
just because new features etc just can't be sanely handled any other way. 

That said, I think we can make the tracking _easier_. 

> - we move the kernel<->userland to a separate package, where it's
>   maintained such that it can be used by both kernel and userland, and
>   Linux will only build when given a pointer to the location of this
>   package.

No. Quite frankly, I absolutely hate projects that do that. I gave up on
following several interesting projects (mostly media decoding) just
because it was just too damn painful to try to make sure that five
different packages were all in sync, and you couldn't find one site to
just download it all from.

So I want the kernel to be as stand-alone as humanly possible. There are a
few dependencies I can't avoid, notably the compiler, and there the kernel
triest to be as permissive as humanly reasonable, and we've always tried
to keep the required tools down to a manageable level (ie even when people
were clamorign for graphical configurations etc, the kernel always made
that very optional indeed).

  ***

Now, on trying to make the tracking _easier_, I would not mind at all to 
move (well-defined) things around a bit to make it clearer what is 
actually exported to user space. But on the other hand, I don't think it's 
actively wrong either to just "mark" them in the headers some way, and 
have tools to extract it automatically _without_ having to separate them 
into some magic location.

In fact, in many ways I'd prefer to have source-level annotations like 
"this is exported to user space" over trying to gather things in one 
place. It would need to have some automated checking ability (that 
probably most people wouldn't run, and would break every once in a while, 
but hey, that's nothing new. That's how "checkconfig" etc works).

But it doesn't have to be "one or the other". I think we could annotate
some things that are nasty to export (and quite frankly, it's not like
this is something "new"; __KERNEL__ is really nothing but a stupid sort of
an "annotation" already, except it's likely the wrong way around: we
should not annotate the stuff that is kernel-only, we should annotate the
stuff that is user-visible, since that should be the exception rather than
the rule).

But even if we decide to have part of the ABI annotated, part of it could 
be split out better. For example, I think the "posix_types.h" split ended 
up working really quite well, and it cleaned up a _huge_ number of type 
issues that people have probably already forgotten because that split was 
done so long ago.

So we _have_ had great success with the "split out specific things"  
before, and I think we should do it whenever there is an obvious chance,
and something is abstracted out and well-defined enough that we can do so.

But in general, I do kind of like the explicit marking. The same way we 
explicitly mark the functions inside the kernel that we expose to modules, 
we could try to mark the data structures and values that we expose to user 
space. That tends to "work".

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-25 15:13 [RFC] Splitting kernel headers and deprecating __KERNEL__ David Howells
                   ` (9 preceding siblings ...)
  2004-11-29 11:08 ` David Howells
@ 2004-11-29 20:01 ` H. Peter Anvin
  2004-11-30  0:34   ` Kyle Moffett
  2004-12-14  7:07   ` Werner Almesberger
  10 siblings, 2 replies; 185+ messages in thread
From: H. Peter Anvin @ 2004-11-29 20:01 UTC (permalink / raw)
  To: linux-kernel

Followup to:  <19865.1101395592@redhat.com>
By author:    David Howells <dhowells@redhat.com>
In newsgroup: linux.dev.kernel
> 
> What we've come up with is this:
> 
>  (1) Create new directories in the linux sources to shadow existing include
>      directories:
> 
> 	NEW DIRECTORY		DIRECTORY SHADOWED
> 	=============		==================
> 	include/user/		include/linux/
> 	include/user-*/		include/asm-*/
> 
>      Note that this doesn't take account of the other directories under
>      include/, but I don't think they're relevant.
> 

I'm not sure if user is a good choice, and user-* is even worse.  Most
people seem to have suggested include/linux-abi for this; I would
personally prefer include/linux-abi/arch for the second.

>  (2) Take each file from the shadowed directory. If it has any userspace
>      relevant stuff, then:
> 
>      (a) Transfer this stuff into a file of the same name in the new
> 	 directory. So, for example, the syscall number list from
> 	 include/asm-i386/unistd.h will be transferred to
> 	 include/user-i386/unistd.h.

I'm not sure you can do such a 1:1 mapping.  In fact, there are cases
where you definitely don't want to, because the current structure
doesn't make much sense.

> 
>      (b) Make kernel file #include the user file. So:
> 
> 		[include/asm-i386/unistd.h]
> 		...
> 		#include <user-i386/unistd.h>
> 		...

Good...

>      (c) Where a user header file requires something from another header file
> 	 (such as a type), that file should include a suitable user header file
> 	 directly:
> 
> 		[include/user-i386/termio.h]
> 		...
> 		#include <user/types.h>
> 		...

Good...

>      (d) stdint types should be used where possible.
> 
> 		[include/user-i386/termios.h]
> 		struct winsize {
> 			uint16_t ws_row;
> 			uint16_t ws_col;
> 			uint16_t ws_xpixel;
> 			uint16_t ws_ypixel;
> 		};

Good, except your "struct winsize" is bad; you're stepping on
namespace which belongs to userspace.  Since we can't use typedefs on
struct tags, I suggest:

struct __kstruct_winsize {
       /* ... */
};

.. and userspace can do:

#define __kstruct_winsize winsize

>      (e) These header files should be bounded with __USER_XXXXX_H conditionals:
> 
> 		[include/user-i386/termios.h]
> 		#ifndef __USER_I386_TERMIOS_H
> 		#define __USER_I386_TERMIOS_H
> 		...
> 		#endif /*  __USER_I386_TERMIOS_H */

Good...

>  (3) Remove all #if(n)def __KERNEL__ clauses.
> 
>  (4) Remove the -D__KERNEL__ from the master kernel Makefile.

Bad!  There is code in the kernel which can compile in userspace for
testing.  This is highly valuable and should be kept.

	-hpa

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29 17:41             ` Linus Torvalds
@ 2004-11-29 22:44               ` Al Viro
  2004-12-01 11:36                 ` Roman Zippel
       [not found]               ` <oract0thnj.fsf@livre.redhat.lsd.ic.unicamp.br>
  2004-11-30 15:35               ` David Woodhouse
  2 siblings, 1 reply; 185+ messages in thread
From: Al Viro @ 2004-11-29 22:44 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Alexandre Oliva, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, David Howells, hch, linux-kernel, libc-hacker,
	zippel

	Speaking of include/* cleanups, here's a work of about 10 minutes
(and yes, result does build):

*	private stuff moved into fs/affs/affs.h
*	include/linux/affs_fs_{i,sb}.h removed, since they do not contain
anything that might be used outside of fs/affs/*
*	includes in fs/affs/* trimmed (as the matter of fact, only super.c
needed something not pulled by affs.h)
*	duplicate MODULE_LICENSE removed from fs/affs/inode.c (fs/affs/super.c
already has exact same thing)
*	bogus extern (affs_fs()) removed (used to be in affs_fs.h, function
in question simply doesn't exist)

That leaves affs_fs.h containing only AFFS_SUPER_MAGIC declaration and
amigaffs.h also seriously cleaned up; it still has a bunch of defines that
are almost certainly never used in userland (they refer to ->b_data and
in-core affs superblock), but these might in theory be useful (similar
stuff for ext2 *is* used by userland code).

If Roman is OK with their removal, AFFS_DATA and friends should also get
moved to fs/affs/affs.h.

Signed-off-by: Al Viro <viro@parcelfarce.linux.theplanet.co.uk>
----
diff -urN RC10-rc2-bk12-base/fs/affs/affs.h RC10-rc2-bk12-current/fs/affs/affs.h
--- RC10-rc2-bk12-base/fs/affs/affs.h	1969-12-31 19:00:00.000000000 -0500
+++ RC10-rc2-bk12-current/fs/affs/affs.h	2004-11-29 17:20:17.536489880 -0500
@@ -0,0 +1,283 @@
+#include <linux/types.h>
+#include <linux/fs.h>
+#include <linux/buffer_head.h>
+#include <linux/affs_fs.h>
+#include <linux/amigaffs.h>
+
+#define AFFS_CACHE_SIZE		PAGE_SIZE
+
+#define AFFS_MAX_PREALLOC	32
+#define AFFS_LC_SIZE		(AFFS_CACHE_SIZE/sizeof(u32)/2)
+#define AFFS_AC_SIZE		(AFFS_CACHE_SIZE/sizeof(struct affs_ext_key)/2)
+#define AFFS_AC_MASK		(AFFS_AC_SIZE-1)
+
+struct affs_ext_key {
+	u32	ext;				/* idx of the extended block */
+	u32	key;				/* block number */
+};
+
+/*
+ * affs fs inode data in memory
+ */
+struct affs_inode_info {
+	u32	 i_opencnt;
+	struct semaphore i_link_lock;		/* Protects internal inode access. */
+	struct semaphore i_ext_lock;		/* Protects internal inode access. */
+#define i_hash_lock i_ext_lock
+	u32	 i_blkcnt;			/* block count */
+	u32	 i_extcnt;			/* extended block count */
+	u32	*i_lc;				/* linear cache of extended blocks */
+	u32	 i_lc_size;
+	u32	 i_lc_shift;
+	u32	 i_lc_mask;
+	struct affs_ext_key *i_ac;		/* associative cache of extended blocks */
+	u32	 i_ext_last;			/* last accessed extended block */
+	struct buffer_head *i_ext_bh;		/* bh of last extended block */
+	loff_t	 mmu_private;
+	u32	 i_protect;			/* unused attribute bits */
+	u32	 i_lastalloc;			/* last allocated block */
+	int	 i_pa_cnt;			/* number of preallocated blocks */
+#if 0
+	s32	 i_original;			/* if != 0, this is the key of the original */
+	u32	 i_data[AFFS_MAX_PREALLOC];	/* preallocated blocks */
+	int	 i_cache_users;			/* Cache cannot be freed while > 0 */
+	unsigned char i_hlink;			/* This is a fake */
+	unsigned char i_pad;
+	s32	 i_parent;			/* parent ino */
+#endif
+	struct inode vfs_inode;
+};
+
+/* short cut to get to the affs specific inode data */
+static inline struct affs_inode_info *AFFS_I(struct inode *inode)
+{
+	return list_entry(inode, struct affs_inode_info, vfs_inode);
+}
+
+/*
+ * super-block data in memory
+ *
+ * Block numbers are adjusted for their actual size
+ *
+ */
+
+struct affs_bm_info {
+	u32 bm_key;			/* Disk block number */
+	u32 bm_free;			/* Free blocks in here */
+};
+
+struct affs_sb_info {
+	int s_partition_size;		/* Partition size in blocks. */
+	int s_reserved;			/* Number of reserved blocks. */
+	//u32 s_blksize;			/* Initial device blksize */
+	u32 s_data_blksize;		/* size of the data block w/o header */
+	u32 s_root_block;		/* FFS root block number. */
+	int s_hashsize;			/* Size of hash table. */
+	unsigned long s_flags;		/* See below. */
+	uid_t s_uid;			/* uid to override */
+	gid_t s_gid;			/* gid to override */
+	umode_t s_mode;			/* mode to override */
+	struct buffer_head *s_root_bh;	/* Cached root block. */
+	struct semaphore s_bmlock;	/* Protects bitmap access. */
+	struct affs_bm_info *s_bitmap;	/* Bitmap infos. */
+	u32 s_bmap_count;		/* # of bitmap blocks. */
+	u32 s_bmap_bits;		/* # of bits in one bitmap blocks */
+	u32 s_last_bmap;
+	struct buffer_head *s_bmap_bh;
+	char *s_prefix;			/* Prefix for volumes and assigns. */
+	int s_prefix_len;		/* Length of prefix. */
+	char s_volume[32];		/* Volume prefix for absolute symlinks. */
+};
+
+#define SF_INTL		0x0001		/* International filesystem. */
+#define SF_BM_VALID	0x0002		/* Bitmap is valid. */
+#define SF_IMMUTABLE	0x0004		/* Protection bits cannot be changed */
+#define SF_QUIET	0x0008		/* chmod errors will be not reported */
+#define SF_SETUID	0x0010		/* Ignore Amiga uid */
+#define SF_SETGID	0x0020		/* Ignore Amiga gid */
+#define SF_SETMODE	0x0040		/* Ignore Amiga protection bits */
+#define SF_MUFS		0x0100		/* Use MUFS uid/gid mapping */
+#define SF_OFS		0x0200		/* Old filesystem */
+#define SF_PREFIX	0x0400		/* Buffer for prefix is allocated */
+#define SF_VERBOSE	0x0800		/* Talk about fs when mounting */
+
+/* short cut to get to the affs specific sb data */
+static inline struct affs_sb_info *AFFS_SB(struct super_block *sb)
+{
+	return sb->s_fs_info;
+}
+
+/* amigaffs.c */
+
+extern int	affs_insert_hash(struct inode *inode, struct buffer_head *bh);
+extern int	affs_remove_hash(struct inode *dir, struct buffer_head *rem_bh);
+extern int	affs_remove_header(struct dentry *dentry);
+extern u32	affs_checksum_block(struct super_block *sb, struct buffer_head *bh);
+extern void	affs_fix_checksum(struct super_block *sb, struct buffer_head *bh);
+extern void	secs_to_datestamp(time_t secs, struct affs_date *ds);
+extern mode_t	prot_to_mode(u32 prot);
+extern void	mode_to_prot(struct inode *inode);
+extern void	affs_error(struct super_block *sb, const char *function, const char *fmt, ...);
+extern void	affs_warning(struct super_block *sb, const char *function, const char *fmt, ...);
+extern int	affs_check_name(const unsigned char *name, int len);
+extern int	affs_copy_name(unsigned char *bstr, struct dentry *dentry);
+
+/* bitmap. c */
+
+extern u32	affs_count_free_blocks(struct super_block *s);
+extern void	affs_free_block(struct super_block *sb, u32 block);
+extern u32	affs_alloc_block(struct inode *inode, u32 goal);
+extern int	affs_init_bitmap(struct super_block *sb, int *flags);
+extern void	affs_free_bitmap(struct super_block *sb);
+
+/* namei.c */
+
+extern int	affs_hash_name(struct super_block *sb, const u8 *name, unsigned int len);
+extern struct dentry *affs_lookup(struct inode *dir, struct dentry *dentry, struct nameidata *);
+extern int	affs_unlink(struct inode *dir, struct dentry *dentry);
+extern int	affs_create(struct inode *dir, struct dentry *dentry, int mode, struct nameidata *);
+extern int	affs_mkdir(struct inode *dir, struct dentry *dentry, int mode);
+extern int	affs_rmdir(struct inode *dir, struct dentry *dentry);
+extern int	affs_link(struct dentry *olddentry, struct inode *dir,
+			  struct dentry *dentry);
+extern int	affs_symlink(struct inode *dir, struct dentry *dentry,
+			     const char *symname);
+extern int	affs_rename(struct inode *old_dir, struct dentry *old_dentry,
+			    struct inode *new_dir, struct dentry *new_dentry);
+
+/* inode.c */
+
+extern unsigned long		 affs_parent_ino(struct inode *dir);
+extern struct inode		*affs_new_inode(struct inode *dir);
+extern int			 affs_notify_change(struct dentry *dentry, struct iattr *attr);
+extern void			 affs_put_inode(struct inode *inode);
+extern void			 affs_delete_inode(struct inode *inode);
+extern void			 affs_clear_inode(struct inode *inode);
+extern void			 affs_read_inode(struct inode *inode);
+extern int			 affs_write_inode(struct inode *inode, int);
+extern int			 affs_add_entry(struct inode *dir, struct inode *inode, struct dentry *dentry, s32 type);
+
+/* file.c */
+
+void		affs_free_prealloc(struct inode *inode);
+extern void	affs_truncate(struct inode *);
+
+/* dir.c */
+
+extern void   affs_dir_truncate(struct inode *);
+
+/* jump tables */
+
+extern struct inode_operations	 affs_file_inode_operations;
+extern struct inode_operations	 affs_dir_inode_operations;
+extern struct inode_operations   affs_symlink_inode_operations;
+extern struct file_operations	 affs_file_operations;
+extern struct file_operations	 affs_file_operations_ofs;
+extern struct file_operations	 affs_dir_operations;
+extern struct address_space_operations	 affs_symlink_aops;
+extern struct address_space_operations	 affs_aops;
+extern struct address_space_operations	 affs_aops_ofs;
+
+extern struct dentry_operations	 affs_dentry_operations;
+extern struct dentry_operations	 affs_dentry_operations_intl;
+
+static inline void
+affs_set_blocksize(struct super_block *sb, int size)
+{
+	sb_set_blocksize(sb, size);
+}
+static inline struct buffer_head *
+affs_bread(struct super_block *sb, int block)
+{
+	pr_debug("affs_bread: %d\n", block);
+	if (block >= AFFS_SB(sb)->s_reserved && block < AFFS_SB(sb)->s_partition_size)
+		return sb_bread(sb, block);
+	return NULL;
+}
+static inline struct buffer_head *
+affs_getblk(struct super_block *sb, int block)
+{
+	pr_debug("affs_getblk: %d\n", block);
+	if (block >= AFFS_SB(sb)->s_reserved && block < AFFS_SB(sb)->s_partition_size)
+		return sb_getblk(sb, block);
+	return NULL;
+}
+static inline struct buffer_head *
+affs_getzeroblk(struct super_block *sb, int block)
+{
+	struct buffer_head *bh;
+	pr_debug("affs_getzeroblk: %d\n", block);
+	if (block >= AFFS_SB(sb)->s_reserved && block < AFFS_SB(sb)->s_partition_size) {
+		bh = sb_getblk(sb, block);
+		lock_buffer(bh);
+		memset(bh->b_data, 0 , sb->s_blocksize);
+		set_buffer_uptodate(bh);
+		unlock_buffer(bh);
+		return bh;
+	}
+	return NULL;
+}
+static inline struct buffer_head *
+affs_getemptyblk(struct super_block *sb, int block)
+{
+	struct buffer_head *bh;
+	pr_debug("affs_getemptyblk: %d\n", block);
+	if (block >= AFFS_SB(sb)->s_reserved && block < AFFS_SB(sb)->s_partition_size) {
+		bh = sb_getblk(sb, block);
+		wait_on_buffer(bh);
+		set_buffer_uptodate(bh);
+		return bh;
+	}
+	return NULL;
+}
+static inline void
+affs_brelse(struct buffer_head *bh)
+{
+	if (bh)
+		pr_debug("affs_brelse: %lld\n", (long long) bh->b_blocknr);
+	brelse(bh);
+}
+
+static inline void
+affs_adjust_checksum(struct buffer_head *bh, u32 val)
+{
+	u32 tmp = be32_to_cpu(((__be32 *)bh->b_data)[5]);
+	((__be32 *)bh->b_data)[5] = cpu_to_be32(tmp - val);
+}
+static inline void
+affs_adjust_bitmapchecksum(struct buffer_head *bh, u32 val)
+{
+	u32 tmp = be32_to_cpu(((__be32 *)bh->b_data)[0]);
+	((__be32 *)bh->b_data)[0] = cpu_to_be32(tmp - val);
+}
+
+static inline void
+affs_lock_link(struct inode *inode)
+{
+	down(&AFFS_I(inode)->i_link_lock);
+}
+static inline void
+affs_unlock_link(struct inode *inode)
+{
+	up(&AFFS_I(inode)->i_link_lock);
+}
+static inline void
+affs_lock_dir(struct inode *inode)
+{
+	down(&AFFS_I(inode)->i_hash_lock);
+}
+static inline void
+affs_unlock_dir(struct inode *inode)
+{
+	up(&AFFS_I(inode)->i_hash_lock);
+}
+static inline void
+affs_lock_ext(struct inode *inode)
+{
+	down(&AFFS_I(inode)->i_ext_lock);
+}
+static inline void
+affs_unlock_ext(struct inode *inode)
+{
+	up(&AFFS_I(inode)->i_ext_lock);
+}
diff -urN RC10-rc2-bk12-base/fs/affs/amigaffs.c RC10-rc2-bk12-current/fs/affs/amigaffs.c
--- RC10-rc2-bk12-base/fs/affs/amigaffs.c	2004-10-18 17:53:43.000000000 -0400
+++ RC10-rc2-bk12-current/fs/affs/amigaffs.c	2004-11-29 17:00:38.594716272 -0500
@@ -8,14 +8,7 @@
  *  Please send bug reports to: hjw@zvw.de
  */
 
-#include <stdarg.h>
-#include <linux/stat.h>
-#include <linux/time.h>
-#include <linux/affs_fs.h>
-#include <linux/string.h>
-#include <linux/mm.h>
-#include <linux/amigaffs.h>
-#include <linux/buffer_head.h>
+#include "affs.h"
 
 extern struct timezone sys_tz;
 
diff -urN RC10-rc2-bk12-base/fs/affs/bitmap.c RC10-rc2-bk12-current/fs/affs/bitmap.c
--- RC10-rc2-bk12-base/fs/affs/bitmap.c	2004-10-18 17:55:28.000000000 -0400
+++ RC10-rc2-bk12-current/fs/affs/bitmap.c	2004-11-29 17:01:22.855987536 -0500
@@ -7,15 +7,7 @@
  *  block allocation, deallocation, calculation of free space.
  */
 
-#include <linux/time.h>
-#include <linux/affs_fs.h>
-#include <linux/stat.h>
-#include <linux/kernel.h>
-#include <linux/slab.h>
-#include <linux/string.h>
-#include <linux/bitops.h>
-#include <linux/amigaffs.h>
-#include <linux/buffer_head.h>
+#include "affs.h"
 
 /* This is, of course, shamelessly stolen from fs/minix */
 
diff -urN RC10-rc2-bk12-base/fs/affs/dir.c RC10-rc2-bk12-current/fs/affs/dir.c
--- RC10-rc2-bk12-base/fs/affs/dir.c	2004-10-18 17:54:55.000000000 -0400
+++ RC10-rc2-bk12-current/fs/affs/dir.c	2004-11-29 17:02:15.392000848 -0500
@@ -13,17 +13,7 @@
  *
  */
 
-#include <asm/uaccess.h>
-#include <linux/errno.h>
-#include <linux/fs.h>
-#include <linux/kernel.h>
-#include <linux/affs_fs.h>
-#include <linux/stat.h>
-#include <linux/string.h>
-#include <linux/mm.h>
-#include <linux/amigaffs.h>
-#include <linux/smp_lock.h>
-#include <linux/buffer_head.h>
+#include "affs.h"
 
 static int affs_readdir(struct file *, void *, filldir_t);
 
diff -urN RC10-rc2-bk12-base/fs/affs/file.c RC10-rc2-bk12-current/fs/affs/file.c
--- RC10-rc2-bk12-base/fs/affs/file.c	2004-10-18 17:54:08.000000000 -0400
+++ RC10-rc2-bk12-current/fs/affs/file.c	2004-11-29 17:04:02.952649144 -0500
@@ -12,24 +12,7 @@
  *  affs regular file handling primitives
  */
 
-#include <asm/div64.h>
-#include <asm/uaccess.h>
-#include <asm/system.h>
-#include <linux/time.h>
-#include <linux/affs_fs.h>
-#include <linux/fcntl.h>
-#include <linux/kernel.h>
-#include <linux/errno.h>
-#include <linux/slab.h>
-#include <linux/stat.h>
-#include <linux/smp_lock.h>
-#include <linux/dirent.h>
-#include <linux/fs.h>
-#include <linux/amigaffs.h>
-#include <linux/mm.h>
-#include <linux/highmem.h>
-#include <linux/pagemap.h>
-#include <linux/buffer_head.h>
+#include "affs.h"
 
 #if PAGE_SIZE < 4096
 #error PAGE_SIZE must be at least 4096
diff -urN RC10-rc2-bk12-base/fs/affs/inode.c RC10-rc2-bk12-current/fs/affs/inode.c
--- RC10-rc2-bk12-base/fs/affs/inode.c	2004-10-18 17:54:07.000000000 -0400
+++ RC10-rc2-bk12-current/fs/affs/inode.c	2004-11-29 17:11:20.702101112 -0500
@@ -10,26 +10,7 @@
  *  (C) 1991  Linus Torvalds - minix filesystem
  */
 
-#include <asm/div64.h>
-#include <linux/errno.h>
-#include <linux/fs.h>
-#include <linux/slab.h>
-#include <linux/stat.h>
-#include <linux/time.h>
-#include <linux/affs_fs.h>
-#include <linux/kernel.h>
-#include <linux/mm.h>
-#include <linux/string.h>
-#include <linux/genhd.h>
-#include <linux/amigaffs.h>
-#include <linux/major.h>
-#include <linux/blkdev.h>
-#include <linux/init.h>
-#include <linux/smp_lock.h>
-#include <linux/buffer_head.h>
-#include <asm/system.h>
-#include <asm/uaccess.h>
-#include <linux/module.h>
+#include "affs.h"
 
 extern struct inode_operations affs_symlink_inode_operations;
 extern struct timezone sys_tz;
@@ -428,4 +409,3 @@
 	affs_unlock_link(inode);
 	goto done;
 }
-MODULE_LICENSE("GPL");
diff -urN RC10-rc2-bk12-base/fs/affs/namei.c RC10-rc2-bk12-current/fs/affs/namei.c
--- RC10-rc2-bk12-base/fs/affs/namei.c	2004-10-18 17:53:46.000000000 -0400
+++ RC10-rc2-bk12-current/fs/affs/namei.c	2004-11-29 17:15:09.782275632 -0500
@@ -8,23 +8,10 @@
  *  (C) 1991  Linus Torvalds - minix filesystem
  */
 
-#include <linux/time.h>
-#include <linux/affs_fs.h>
-#include <linux/kernel.h>
-#include <linux/string.h>
-#include <linux/stat.h>
-#include <linux/fcntl.h>
-#include <linux/amigaffs.h>
-#include <linux/smp_lock.h>
-#include <linux/buffer_head.h>
-#include <asm/uaccess.h>
-
-#include <linux/errno.h>
+#include "affs.h"
 
 typedef int (*toupper_t)(int);
 
-extern struct inode_operations affs_symlink_inode_operations;
-
 static int	 affs_toupper(int ch);
 static int	 affs_hash_dentry(struct dentry *, struct qstr *);
 static int       affs_compare_dentry(struct dentry *, struct qstr *, struct qstr *);
diff -urN RC10-rc2-bk12-base/fs/affs/super.c RC10-rc2-bk12-current/fs/affs/super.c
--- RC10-rc2-bk12-base/fs/affs/super.c	2004-10-18 17:54:39.000000000 -0400
+++ RC10-rc2-bk12-current/fs/affs/super.c	2004-11-29 17:17:59.947406600 -0500
@@ -11,26 +11,10 @@
  */
 
 #include <linux/module.h>
-#include <linux/errno.h>
-#include <linux/fs.h>
-#include <linux/slab.h>
-#include <linux/stat.h>
-#include <linux/time.h>
-#include <linux/affs_fs.h>
-#include <linux/kernel.h>
-#include <linux/mm.h>
-#include <linux/string.h>
-#include <linux/genhd.h>
-#include <linux/amigaffs.h>
-#include <linux/major.h>
-#include <linux/blkdev.h>
 #include <linux/init.h>
-#include <linux/smp_lock.h>
-#include <linux/buffer_head.h>
-#include <linux/vfs.h>
+#include <linux/statfs.h>
 #include <linux/parser.h>
-#include <asm/system.h>
-#include <asm/uaccess.h>
+#include "affs.h"
 
 extern struct timezone sys_tz;
 
diff -urN RC10-rc2-bk12-base/fs/affs/symlink.c RC10-rc2-bk12-current/fs/affs/symlink.c
--- RC10-rc2-bk12-base/fs/affs/symlink.c	2004-10-18 17:53:42.000000000 -0400
+++ RC10-rc2-bk12-current/fs/affs/symlink.c	2004-11-29 17:14:21.318643224 -0500
@@ -8,14 +8,7 @@
  *  affs symlink handling code
  */
 
-#include <linux/errno.h>
-#include <linux/fs.h>
-#include <linux/stat.h>
-#include <linux/affs_fs.h>
-#include <linux/amigaffs.h>
-#include <linux/pagemap.h>
-#include <linux/smp_lock.h>
-#include <linux/buffer_head.h>
+#include "affs.h"
 
 static int affs_symlink_readpage(struct file *file, struct page *page)
 {
diff -urN RC10-rc2-bk12-base/include/linux/affs_fs.h RC10-rc2-bk12-current/include/linux/affs_fs.h
--- RC10-rc2-bk12-base/include/linux/affs_fs.h	2004-10-18 17:53:06.000000000 -0400
+++ RC10-rc2-bk12-current/include/linux/affs_fs.h	2004-11-29 16:53:07.815245168 -0500
@@ -3,94 +3,5 @@
 /*
  * The affs filesystem constants/structures
  */
-
-#include <linux/types.h>
-
-#include <linux/affs_fs_i.h>
-#include <linux/affs_fs_sb.h>
-
 #define AFFS_SUPER_MAGIC 0xadff
-
-struct affs_date;
-
-/* --- Prototypes -----------------------------------------------------------------------------	*/
-
-/* amigaffs.c */
-
-extern int	affs_insert_hash(struct inode *inode, struct buffer_head *bh);
-extern int	affs_remove_hash(struct inode *dir, struct buffer_head *rem_bh);
-extern int	affs_remove_header(struct dentry *dentry);
-extern u32	affs_checksum_block(struct super_block *sb, struct buffer_head *bh);
-extern void	affs_fix_checksum(struct super_block *sb, struct buffer_head *bh);
-extern void	secs_to_datestamp(time_t secs, struct affs_date *ds);
-extern mode_t	prot_to_mode(u32 prot);
-extern void	mode_to_prot(struct inode *inode);
-extern void	affs_error(struct super_block *sb, const char *function, const char *fmt, ...);
-extern void	affs_warning(struct super_block *sb, const char *function, const char *fmt, ...);
-extern int	affs_check_name(const unsigned char *name, int len);
-extern int	affs_copy_name(unsigned char *bstr, struct dentry *dentry);
-
-/* bitmap. c */
-
-extern u32	affs_count_free_blocks(struct super_block *s);
-extern void	affs_free_block(struct super_block *sb, u32 block);
-extern u32	affs_alloc_block(struct inode *inode, u32 goal);
-extern int	affs_init_bitmap(struct super_block *sb, int *flags);
-extern void	affs_free_bitmap(struct super_block *sb);
-
-/* namei.c */
-
-extern int	affs_hash_name(struct super_block *sb, const u8 *name, unsigned int len);
-extern struct dentry *affs_lookup(struct inode *dir, struct dentry *dentry, struct nameidata *);
-extern int	affs_unlink(struct inode *dir, struct dentry *dentry);
-extern int	affs_create(struct inode *dir, struct dentry *dentry, int mode, struct nameidata *);
-extern int	affs_mkdir(struct inode *dir, struct dentry *dentry, int mode);
-extern int	affs_rmdir(struct inode *dir, struct dentry *dentry);
-extern int	affs_link(struct dentry *olddentry, struct inode *dir,
-			  struct dentry *dentry);
-extern int	affs_symlink(struct inode *dir, struct dentry *dentry,
-			     const char *symname);
-extern int	affs_rename(struct inode *old_dir, struct dentry *old_dentry,
-			    struct inode *new_dir, struct dentry *new_dentry);
-
-/* inode.c */
-
-extern unsigned long		 affs_parent_ino(struct inode *dir);
-extern struct inode		*affs_new_inode(struct inode *dir);
-extern int			 affs_notify_change(struct dentry *dentry, struct iattr *attr);
-extern void			 affs_put_inode(struct inode *inode);
-extern void			 affs_delete_inode(struct inode *inode);
-extern void			 affs_clear_inode(struct inode *inode);
-extern void			 affs_read_inode(struct inode *inode);
-extern int			 affs_write_inode(struct inode *inode, int);
-extern int			 affs_add_entry(struct inode *dir, struct inode *inode, struct dentry *dentry, s32 type);
-
-/* super.c */
-
-extern int			 affs_fs(void);
-
-/* file.c */
-
-void		affs_free_prealloc(struct inode *inode);
-extern void	affs_truncate(struct inode *);
-
-/* dir.c */
-
-extern void   affs_dir_truncate(struct inode *);
-
-/* jump tables */
-
-extern struct inode_operations	 affs_file_inode_operations;
-extern struct inode_operations	 affs_dir_inode_operations;
-extern struct inode_operations   affs_symlink_inode_operations;
-extern struct file_operations	 affs_file_operations;
-extern struct file_operations	 affs_file_operations_ofs;
-extern struct file_operations	 affs_dir_operations;
-extern struct address_space_operations	 affs_symlink_aops;
-extern struct address_space_operations	 affs_aops;
-extern struct address_space_operations	 affs_aops_ofs;
-
-extern struct dentry_operations	 affs_dentry_operations;
-extern struct dentry_operations	 affs_dentry_operations_intl;
-
 #endif
diff -urN RC10-rc2-bk12-base/include/linux/affs_fs_i.h RC10-rc2-bk12-current/include/linux/affs_fs_i.h
--- RC10-rc2-bk12-base/include/linux/affs_fs_i.h	2004-10-18 17:55:28.000000000 -0400
+++ RC10-rc2-bk12-current/include/linux/affs_fs_i.h	1969-12-31 19:00:00.000000000 -0500
@@ -1,59 +0,0 @@
-#ifndef _AFFS_FS_I
-#define _AFFS_FS_I
-
-#include <linux/a.out.h>
-#include <linux/fs.h>
-#include <asm/semaphore.h>
-
-#define AFFS_CACHE_SIZE		PAGE_SIZE
-//#define AFFS_CACHE_SIZE		(4*4)
-
-#define AFFS_MAX_PREALLOC	32
-#define AFFS_LC_SIZE		(AFFS_CACHE_SIZE/sizeof(u32)/2)
-#define AFFS_AC_SIZE		(AFFS_CACHE_SIZE/sizeof(struct affs_ext_key)/2)
-#define AFFS_AC_MASK		(AFFS_AC_SIZE-1)
-
-struct affs_ext_key {
-	u32	ext;				/* idx of the extended block */
-	u32	key;				/* block number */
-};
-
-/*
- * affs fs inode data in memory
- */
-struct affs_inode_info {
-	u32	 i_opencnt;
-	struct semaphore i_link_lock;		/* Protects internal inode access. */
-	struct semaphore i_ext_lock;		/* Protects internal inode access. */
-#define i_hash_lock i_ext_lock
-	u32	 i_blkcnt;			/* block count */
-	u32	 i_extcnt;			/* extended block count */
-	u32	*i_lc;				/* linear cache of extended blocks */
-	u32	 i_lc_size;
-	u32	 i_lc_shift;
-	u32	 i_lc_mask;
-	struct affs_ext_key *i_ac;		/* associative cache of extended blocks */
-	u32	 i_ext_last;			/* last accessed extended block */
-	struct buffer_head *i_ext_bh;		/* bh of last extended block */
-	loff_t	 mmu_private;
-	u32	 i_protect;			/* unused attribute bits */
-	u32	 i_lastalloc;			/* last allocated block */
-	int	 i_pa_cnt;			/* number of preallocated blocks */
-#if 0
-	s32	 i_original;			/* if != 0, this is the key of the original */
-	u32	 i_data[AFFS_MAX_PREALLOC];	/* preallocated blocks */
-	int	 i_cache_users;			/* Cache cannot be freed while > 0 */
-	unsigned char i_hlink;			/* This is a fake */
-	unsigned char i_pad;
-	s32	 i_parent;			/* parent ino */
-#endif
-	struct inode vfs_inode;
-};
-
-/* short cut to get to the affs specific inode data */
-static inline struct affs_inode_info *AFFS_I(struct inode *inode)
-{
-	return list_entry(inode, struct affs_inode_info, vfs_inode);
-}
-
-#endif
diff -urN RC10-rc2-bk12-base/include/linux/affs_fs_sb.h RC10-rc2-bk12-current/include/linux/affs_fs_sb.h
--- RC10-rc2-bk12-base/include/linux/affs_fs_sb.h	2004-10-18 17:53:22.000000000 -0400
+++ RC10-rc2-bk12-current/include/linux/affs_fs_sb.h	1969-12-31 19:00:00.000000000 -0500
@@ -1,57 +0,0 @@
-#ifndef _AFFS_FS_SB
-#define _AFFS_FS_SB
-
-/*
- * super-block data in memory
- *
- * Block numbers are adjusted for their actual size
- *
- */
-
-struct affs_bm_info {
-	u32 bm_key;			/* Disk block number */
-	u32 bm_free;			/* Free blocks in here */
-};
-
-struct affs_sb_info {
-	int s_partition_size;		/* Partition size in blocks. */
-	int s_reserved;			/* Number of reserved blocks. */
-	//u32 s_blksize;			/* Initial device blksize */
-	u32 s_data_blksize;		/* size of the data block w/o header */
-	u32 s_root_block;		/* FFS root block number. */
-	int s_hashsize;			/* Size of hash table. */
-	unsigned long s_flags;		/* See below. */
-	uid_t s_uid;			/* uid to override */
-	gid_t s_gid;			/* gid to override */
-	umode_t s_mode;			/* mode to override */
-	struct buffer_head *s_root_bh;	/* Cached root block. */
-	struct semaphore s_bmlock;	/* Protects bitmap access. */
-	struct affs_bm_info *s_bitmap;	/* Bitmap infos. */
-	u32 s_bmap_count;		/* # of bitmap blocks. */
-	u32 s_bmap_bits;		/* # of bits in one bitmap blocks */
-	u32 s_last_bmap;
-	struct buffer_head *s_bmap_bh;
-	char *s_prefix;			/* Prefix for volumes and assigns. */
-	int s_prefix_len;		/* Length of prefix. */
-	char s_volume[32];		/* Volume prefix for absolute symlinks. */
-};
-
-#define SF_INTL		0x0001		/* International filesystem. */
-#define SF_BM_VALID	0x0002		/* Bitmap is valid. */
-#define SF_IMMUTABLE	0x0004		/* Protection bits cannot be changed */
-#define SF_QUIET	0x0008		/* chmod errors will be not reported */
-#define SF_SETUID	0x0010		/* Ignore Amiga uid */
-#define SF_SETGID	0x0020		/* Ignore Amiga gid */
-#define SF_SETMODE	0x0040		/* Ignore Amiga protection bits */
-#define SF_MUFS		0x0100		/* Use MUFS uid/gid mapping */
-#define SF_OFS		0x0200		/* Old filesystem */
-#define SF_PREFIX	0x0400		/* Buffer for prefix is allocated */
-#define SF_VERBOSE	0x0800		/* Talk about fs when mounting */
-
-/* short cut to get to the affs specific sb data */
-static inline struct affs_sb_info *AFFS_SB(struct super_block *sb)
-{
-	return sb->s_fs_info;
-}
-
-#endif
diff -urN RC10-rc2-bk12-base/include/linux/amigaffs.h RC10-rc2-bk12-current/include/linux/amigaffs.h
--- RC10-rc2-bk12-base/include/linux/amigaffs.h	2004-10-18 17:53:23.000000000 -0400
+++ RC10-rc2-bk12-current/include/linux/amigaffs.h	2004-11-29 17:20:01.572916712 -0500
@@ -2,8 +2,6 @@
 #define AMIGAFFS_H
 
 #include <linux/types.h>
-#include <linux/buffer_head.h>
-#include <linux/string.h>
 #include <asm/byteorder.h>
 
 /* AmigaOS allows file names with up to 30 characters length.
@@ -20,107 +18,6 @@
 #define AFFS_GET_HASHENTRY(data,hashkey) be32_to_cpu(((struct dir_front *)data)->hashtable[hashkey])
 #define AFFS_BLOCK(sb, bh, blk)		(AFFS_HEAD(bh)->table[AFFS_SB(sb)->s_hashsize-1-(blk)])
 
-static inline void
-affs_set_blocksize(struct super_block *sb, int size)
-{
-	sb_set_blocksize(sb, size);
-}
-static inline struct buffer_head *
-affs_bread(struct super_block *sb, int block)
-{
-	pr_debug("affs_bread: %d\n", block);
-	if (block >= AFFS_SB(sb)->s_reserved && block < AFFS_SB(sb)->s_partition_size)
-		return sb_bread(sb, block);
-	return NULL;
-}
-static inline struct buffer_head *
-affs_getblk(struct super_block *sb, int block)
-{
-	pr_debug("affs_getblk: %d\n", block);
-	if (block >= AFFS_SB(sb)->s_reserved && block < AFFS_SB(sb)->s_partition_size)
-		return sb_getblk(sb, block);
-	return NULL;
-}
-static inline struct buffer_head *
-affs_getzeroblk(struct super_block *sb, int block)
-{
-	struct buffer_head *bh;
-	pr_debug("affs_getzeroblk: %d\n", block);
-	if (block >= AFFS_SB(sb)->s_reserved && block < AFFS_SB(sb)->s_partition_size) {
-		bh = sb_getblk(sb, block);
-		lock_buffer(bh);
-		memset(bh->b_data, 0 , sb->s_blocksize);
-		set_buffer_uptodate(bh);
-		unlock_buffer(bh);
-		return bh;
-	}
-	return NULL;
-}
-static inline struct buffer_head *
-affs_getemptyblk(struct super_block *sb, int block)
-{
-	struct buffer_head *bh;
-	pr_debug("affs_getemptyblk: %d\n", block);
-	if (block >= AFFS_SB(sb)->s_reserved && block < AFFS_SB(sb)->s_partition_size) {
-		bh = sb_getblk(sb, block);
-		wait_on_buffer(bh);
-		set_buffer_uptodate(bh);
-		return bh;
-	}
-	return NULL;
-}
-static inline void
-affs_brelse(struct buffer_head *bh)
-{
-	if (bh)
-		pr_debug("affs_brelse: %lld\n", (long long) bh->b_blocknr);
-	brelse(bh);
-}
-
-static inline void
-affs_adjust_checksum(struct buffer_head *bh, u32 val)
-{
-	u32 tmp = be32_to_cpu(((__be32 *)bh->b_data)[5]);
-	((__be32 *)bh->b_data)[5] = cpu_to_be32(tmp - val);
-}
-static inline void
-affs_adjust_bitmapchecksum(struct buffer_head *bh, u32 val)
-{
-	u32 tmp = be32_to_cpu(((__be32 *)bh->b_data)[0]);
-	((__be32 *)bh->b_data)[0] = cpu_to_be32(tmp - val);
-}
-
-static inline void
-affs_lock_link(struct inode *inode)
-{
-	down(&AFFS_I(inode)->i_link_lock);
-}
-static inline void
-affs_unlock_link(struct inode *inode)
-{
-	up(&AFFS_I(inode)->i_link_lock);
-}
-static inline void
-affs_lock_dir(struct inode *inode)
-{
-	down(&AFFS_I(inode)->i_hash_lock);
-}
-static inline void
-affs_unlock_dir(struct inode *inode)
-{
-	up(&AFFS_I(inode)->i_hash_lock);
-}
-static inline void
-affs_lock_ext(struct inode *inode)
-{
-	down(&AFFS_I(inode)->i_ext_lock);
-}
-static inline void
-affs_unlock_ext(struct inode *inode)
-{
-	up(&AFFS_I(inode)->i_ext_lock);
-}
-
 #ifdef __LITTLE_ENDIAN
 #define BO_EXBITS	0x18UL
 #elif defined(__BIG_ENDIAN)

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
       [not found]               ` <oract0thnj.fsf@livre.redhat.lsd.ic.unicamp.br>
@ 2004-11-29 23:00                 ` Linus Torvalds
  2004-11-30  0:45                   ` H. Peter Anvin
                                     ` (3 more replies)
  0 siblings, 4 replies; 185+ messages in thread
From: Linus Torvalds @ 2004-11-29 23:00 UTC (permalink / raw)
  To: Alexandre Oliva
  Cc: Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox,
	David Howells, hch, linux-kernel, libc-hacker



On Mon, 29 Nov 2004, Alexandre Oliva wrote:
> 
> I don't see it as obvious at all.  The need for an agreement between
> two parties on an ABI doesn't imply that one party gets to define it
> and the other gets to follow it. 

Sorry, but that's not how it works.

He who writes the code decides what it is. In this case, if the kernel 
does a new extension, it's the kernel that gets to decide what it is. 
Full stop.

If glibc wants to do something new, go wild. The kernel won't care.

And that's really the fundamental issue. The kernel does not care what
user land does. The kernel exports functionality, the kernel does _not_
ask user land to help.

That _does_ make it a one-way street. Sorry.

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29 20:01 ` H. Peter Anvin
@ 2004-11-30  0:34   ` Kyle Moffett
  2004-11-30  0:51     ` H. Peter Anvin
  2004-12-14  7:07   ` Werner Almesberger
  1 sibling, 1 reply; 185+ messages in thread
From: Kyle Moffett @ 2004-11-30  0:34 UTC (permalink / raw)
  To: H. Peter Anvin; +Cc: linux-kernel

On Nov 29, 2004, at 15:01, H. Peter Anvin wrote:
> Most people seem to have suggested include/linux-abi for this; I would
> personally prefer include/linux-abi/arch for the second.

It seems like there are two ways to adjust the headers.  We could move
the private headers to a new directory (include/kernel?), or we could 
move
the public headers to a new directory (include/linux-abi?).  I am 
willing to
work on some simple and trivial patches to begin doing either one, if we
can reach an agreement as to which one is preferrable (and what to call
the new directory.)

>>      (a) Transfer this stuff into a file of the same name in the new
>> 	 directory. So, for example, the syscall number list from
>> 	 include/asm-i386/unistd.h will be transferred to
>> 	 include/user-i386/unistd.h.
> I'm not sure you can do such a 1:1 mapping.  In fact, there are cases
> where you definitely don't want to, because the current structure
> doesn't make much sense.

I think that the initial trivial patches would probably preserve a 1:1
mapping, just for compatibility.  On the other hand, future patches 
might
rearrange things to make more logical sense.

>>      (d) stdint types should be used where possible.
>>
>> 		[include/user-i386/termios.h]
>> 		struct winsize {
>> 			uint16_t ws_row;
>> 			uint16_t ws_col;
>> 			uint16_t ws_xpixel;
>> 			uint16_t ws_ypixel;
>> 		};
> Good, except your "struct winsize" is bad; you're stepping on
> namespace which belongs to userspace.  Since we can't use typedefs on
> struct tags, I suggest:
>
> struct __kstruct_winsize {
>        /* ... */
> };
>
> .. and userspace can do:
>
> #define __kstruct_winsize winsize

My initial suggestion would be to leave the types as-is, primarily for
the reasons in Linus' earlier email, but also for simplicity and to
prevent inadvertent breakage.

>>  (3) Remove all #if(n)def __KERNEL__ clauses.
>>
>>  (4) Remove the -D__KERNEL__ from the master kernel Makefile.
>
> Bad!  There is code in the kernel which can compile in userspace for
> testing.  This is highly valuable and should be kept.

I would propose that if we decide to move the public headers instead of
the internal kernel headers, we autogenerate headers for installation
that have a #warning wrapper if __KERNEL__ isn't defined.

Cheers,
Kyle Moffett

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCM/CS/IT/U d- s++: a17 C++++>$ UB/L/X/*++++(+)>$ P+++(++++)>$
L++++(+++) E W++(+) N+++(++) o? K? w--- O? M++ V? PS+() PE+(-) Y+
PGP+++ t+(+++) 5 X R? tv-(--) b++++(++) DI+ D+ G e->++++$ h!*()>++$ r  
!y?(-)
------END GEEK CODE BLOCK------



^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29 23:00                 ` Linus Torvalds
@ 2004-11-30  0:45                   ` H. Peter Anvin
  2004-11-30  5:31                   ` Herbert Poetzl
                                     ` (2 subsequent siblings)
  3 siblings, 0 replies; 185+ messages in thread
From: H. Peter Anvin @ 2004-11-30  0:45 UTC (permalink / raw)
  To: linux-kernel

Followup to:  <Pine.LNX.4.58.0411291458040.22796@ppc970.osdl.org>
By author:    Linus Torvalds <torvalds@osdl.org>
In newsgroup: linux.dev.kernel
> 
> If glibc wants to do something new, go wild. The kernel won't care.
> 
> And that's really the fundamental issue. The kernel does not care what
> user land does. The kernel exports functionality, the kernel does _not_
> ask user land to help.
> 
> That _does_ make it a one-way street. Sorry.
> 

And it SHOULD be a one-way street.  A lot of the ugliness in the
current stuff comes from the fact that the kernel tries to export
libc4/5 internals, instead of the real kernel ABI.

	-hpa

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30  0:34   ` Kyle Moffett
@ 2004-11-30  0:51     ` H. Peter Anvin
  0 siblings, 0 replies; 185+ messages in thread
From: H. Peter Anvin @ 2004-11-30  0:51 UTC (permalink / raw)
  To: linux-kernel

Followup to:  <A697CF5A-4267-11D9-8DB8-000393ACC76E@mac.com>
By author:    Kyle Moffett <mrmacman_g4@mac.com>
In newsgroup: linux.dev.kernel
>
> On Nov 29, 2004, at 15:01, H. Peter Anvin wrote:
> > Most people seem to have suggested include/linux-abi for this; I would
> > personally prefer include/linux-abi/arch for the second.
> 
> It seems like there are two ways to adjust the headers.  We could move
> the private headers to a new directory (include/kernel?), or we could 
> move
> the public headers to a new directory (include/linux-abi?).  I am 
> willing to
> work on some simple and trivial patches to begin doing either one, if we
> can reach an agreement as to which one is preferrable (and what to call
> the new directory.)
> 

If we can preserve compatibility, moving the private headers to a
separate place makes sense.  However, preserving compatibility is not
a good thing, because a lot of the brokenness is in how things are
exported.  Thus, you want libc to be able to have a linux/* wrapper
for backwards compatibility.

My first choice would be to put the ABI headers in linux/abi.

> > Good, except your "struct winsize" is bad; you're stepping on
> > namespace which belongs to userspace.  Since we can't use typedefs on
> > struct tags, I suggest:
> >
> > struct __kstruct_winsize {
> >        /* ... */
> > };
> >
> > .. and userspace can do:
> >
> > #define __kstruct_winsize winsize
> 
> My initial suggestion would be to leave the types as-is, primarily for
> the reasons in Linus' earlier email, but also for simplicity and to
> prevent inadvertent breakage.

True; I hadn't considered the weirdness of the namespace pollution
issue.  However, the issue with struct tags remain.

Note also that not all ABI issues can be well-described in C headers,
and certainly not the way it currently is.  Syscall numbers and ioctl
APIs are examples there (I do some hideously ugly hacks in klibc to
try to auto-derive as much of the ABI as possible; this is part of
making klibc very easy to port to new architectures and keeps
maintenance in one place.)  However, cleaning up the headers is a
major step forward.

> >>  (3) Remove all #if(n)def __KERNEL__ clauses.
> >>
> >>  (4) Remove the -D__KERNEL__ from the master kernel Makefile.
> >
> > Bad!  There is code in the kernel which can compile in userspace for
> > testing.  This is highly valuable and should be kept.
> 
> I would propose that if we decide to move the public headers instead of
> the internal kernel headers, we autogenerate headers for installation
> that have a #warning wrapper if __KERNEL__ isn't defined.

	-hpa

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29 23:00                 ` Linus Torvalds
  2004-11-30  0:45                   ` H. Peter Anvin
@ 2004-11-30  5:31                   ` Herbert Poetzl
  2004-11-30  6:44                   ` bert hubert
  2004-11-30 21:22                   ` Alexandre Oliva
  3 siblings, 0 replies; 185+ messages in thread
From: Herbert Poetzl @ 2004-11-30  5:31 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Alexandre Oliva, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, David Howells, hch, linux-kernel, libc-hacker

On Mon, Nov 29, 2004 at 03:00:17PM -0800, Linus Torvalds wrote:
> 
> 
> On Mon, 29 Nov 2004, Alexandre Oliva wrote:
> > 
> > I don't see it as obvious at all.  The need for an agreement between
> > two parties on an ABI doesn't imply that one party gets to define it
> > and the other gets to follow it. 
> 
> Sorry, but that's not how it works.
> 
> He who writes the code decides what it is. In this case, if the kernel 
> does a new extension, it's the kernel that gets to decide what it is. 
> Full stop.
> 
> If glibc wants to do something new, go wild. The kernel won't care.
> 
> And that's really the fundamental issue. The kernel does not care what
> user land does. The kernel exports functionality, the kernel does _not_
> ask user land to help.

except for the user land kernel helpers ;)
via call_usermodehelper()

> That _does_ make it a one-way street. Sorry.
> 
> 		Linus

sorry too,
Herbert

> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29 23:00                 ` Linus Torvalds
  2004-11-30  0:45                   ` H. Peter Anvin
  2004-11-30  5:31                   ` Herbert Poetzl
@ 2004-11-30  6:44                   ` bert hubert
  2004-11-30 21:22                   ` Alexandre Oliva
  3 siblings, 0 replies; 185+ messages in thread
From: bert hubert @ 2004-11-30  6:44 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Alexandre Oliva, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, David Howells, hch, linux-kernel, libc-hacker

On Mon, Nov 29, 2004 at 03:00:17PM -0800, Linus Torvalds wrote:

> And that's really the fundamental issue. The kernel does not care what
> user land does. The kernel exports functionality, the kernel does _not_
> ask user land to help.

Reminds me of the Klingon programming language:

http://tinyurl.com/4rjco

7 : "Klingon function calls do not have 'parameters' -- they have
'arguments' -- and they ALWAYS WIN THEM."

8 : "What is this talk of 'release'? Klingons do not make software
'releases.' Our software 'escapes' leaving a bloody trail of designers and
quality assurance people in its wake."

9 : "Indentation?! - I will show you how to indent when I indent your
skull!"

-- 
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://lartc.org           Linux Advanced Routing & Traffic Control HOWTO

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29  9:42           ` David Woodhouse
@ 2004-11-30  8:08             ` Alex Riesen
  2004-11-30  8:57               ` Alex Riesen
  2004-12-01 11:46             ` Ralf Baechle
  1 sibling, 1 reply; 185+ messages in thread
From: Alex Riesen @ 2004-11-30  8:08 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Linus Torvalds, Paul Mackerras, Greg KH, Matthew Wilcox,
	David Howells, hch, aoliva, linux-kernel, libc-hacker

On Mon, 29 Nov 2004 09:42:16 +0000, David Woodhouse <dwmw2@infradead.org> wrote:
> I've lost track of the number of times things have broken because of
> incorrect use of kernel headers from userspace. That's what we're trying
> to fix -- by putting only the bits which are _supposed_ to be visible
> into files which userspace sees, where we know they define part of the
> userspace API and hence we can be extremely careful when editing them.

how about generating the user-visible interfaces and structures from the headers
using something like sparse and defined annotations? I.e.:

struct iovec
{
	void __user *iov_base;
	__kernel_size_t iov_len;
} __user_api__;

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30  8:08             ` Alex Riesen
@ 2004-11-30  8:57               ` Alex Riesen
  0 siblings, 0 replies; 185+ messages in thread
From: Alex Riesen @ 2004-11-30  8:57 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Linus Torvalds, Paul Mackerras, Greg KH, Matthew Wilcox,
	David Howells, hch, aoliva, linux-kernel, libc-hacker

I should have read the whole thread before posting. My apologies, Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__ 
  2004-11-26 14:19       ` Matthew Wilcox
  2004-11-26 14:33         ` David Woodhouse
  2004-11-27  3:46         ` Adam Heath
@ 2004-11-30 12:18         ` Horst von Brand
  2 siblings, 0 replies; 185+ messages in thread
From: Horst von Brand @ 2004-11-30 12:18 UTC (permalink / raw)
  To: Matthew Wilcox
  Cc: David Woodhouse, David Howells, Alexandre Oliva, torvalds, hch,
	linux-kernel, libc-alpha

Matthew Wilcox <matthew@wil.cx> said:
> On Fri, Nov 26, 2004 at 12:00:43PM +0000, David Woodhouse wrote:
> > On Fri, 2004-11-26 at 11:58 +0000, David Howells wrote:

> > > How about calling the interface headers "kapi*/" instead of "user*/".
> > > In case you haven't guessed, "kapi" would be short for "kernel-api".

> > I don't think that change really makes any difference. The nomenclature
> > really isn't _that_ important.

> Indeed.  We could also make this transparent to userspace by using a script
> to copy the user-* headers to /usr/include.  Something like this:

And get them promptly out of sync with glibc &c when the kernel changes.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__ 
  2004-11-29 17:09           ` Alexandre Oliva
  2004-11-29 17:41             ` Linus Torvalds
@ 2004-11-30 15:33             ` David Howells
  2004-11-30 15:51               ` Linus Torvalds
  2004-11-30 16:18               ` David Howells
  1 sibling, 2 replies; 185+ messages in thread
From: David Howells @ 2004-11-30 15:33 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Alexandre Oliva, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, hch, linux-kernel, libc-hacker


> But in general, I do kind of like the explicit marking. The same way we 
> explicitly mark the functions inside the kernel that we expose to modules, 
> we could try to mark the data structures and values that we expose to user 
> space. That tends to "work".

That gets trickier when it comes to #defines, I think.

Do you really object that much to splitting the header files as I proposed?
This should obviate the requirement for such marking (because it's implicit in
the location). Getting rid of the #ifdefs would also make the "Thou shalt not
use #ifdef" camp happier.

Doing this really shouldn't break the syscall API/ABI at all; barring of
course the potential change of location within /usr/include/ of the installed
headers, but that is simple to address. All this does is to separate out the
userspace available structures and constants.

I can grant that you might not want to use stdint types because not all the
toolchains that might be used on the kernel support them, but that's not a
reason to throw out the whole proposal.

Personally, I'd prefer us to move to using standard C99 types in lieu of u32
and co at least for the interface to userspace because they are just that:
standard.

However, I can see that there may be objections due to the fact that types may
become different logical sizes, and thus cause warnings in printf-format
string checking and pointer casting.

Also, the comparison you made to modules isn't exactly valid. We annotate
variables and functions that modules are allowed to access, but we never
export these to userspace directly; and we don't annotate structures, inline
functions, constants and macros into which categories the user-visible stuff
falls for use by modules.

David

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29 17:41             ` Linus Torvalds
  2004-11-29 22:44               ` Al Viro
       [not found]               ` <oract0thnj.fsf@livre.redhat.lsd.ic.unicamp.br>
@ 2004-11-30 15:35               ` David Woodhouse
  2004-11-30 15:58                 ` Linus Torvalds
  2004-11-30 16:33                 ` David Howells
  2 siblings, 2 replies; 185+ messages in thread
From: David Woodhouse @ 2004-11-30 15:35 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Alexandre Oliva, Paul Mackerras, Greg KH, Matthew Wilcox,
	David Howells, hch, linux-kernel, libc-hacker

On Mon, 2004-11-29 at 09:41 -0800, Linus Torvalds wrote:
> In fact, in many ways I'd prefer to have source-level annotations like 
> "this is exported to user space" over trying to gather things in one 
> place.

I don't think you would; not once you really tried it. 

That's what the littering of ifdef __KERNEL__ attempts to do, and
there's not really any better way of doing it than that. Whatever form
of annotation you came up with, it wouldn't be much different to using
__KERNEL__ and performing the 'export' step using unifdef. Changing the
sense of it so that we annotate the stuff which _is_ visible rather than
the stuff which _isn't_ doesn't really make it fundamentally different
either.

If we want to make stuff cleaner in userspace that means spreading the
headers much more liberally with such ifdefs or whatever trick we use to
replace them. It gets ugly quite fast -- and we already know that the
fix for having too many ifdefs is to split stuff up more sensibly. 
By splitting the visible parts into separate files we actually help to
make people _think_ when they're exposing stuff to userspace, and it
would help to direct particularly close attention to reviewing
structures therein. For example, if the netfilter people tried to add
this to a header file which is explicitly for userspace consumption,
rather than hidden away in kernel private headers and just _happening_
to be part of the userspace ABI, they'd never get away with it:

struct ipt_rateinfo {
	u_int32_t avg;    /* Average secs between packets * scale */
        u_int32_t burst;  /* Period multiplier for upper limit. */
 
        /* Used internally by the kernel */
        unsigned long prev;
        u_int32_t credit;
        u_int32_t credit_cap, cost;
 
        /* Ugly, ugly fucker. */
        struct ipt_rateinfo *master;
};

Use of an 'unsigned long' which gratuitously changes the structure size
between 32-bit userspace and 64-bit kernel, without even any _reason_
for it being there. That kind of thing would be far more obvious if the
headers are split, and much less so if we continue as we are. 

That and the ugliness of the __KERNEL__ or whatever other annotation
scheme we come up with are the main reasons why I think it wants to be
separate header files for sharing with userspace, rather than continuing
with the mess we have.

Let's go through the kernel headers and identify what actually needs to
be exported; yes. But let's not 'annotate' it and post-process the mess
of header files to sanitise it; let's just take the parts which need to
be shared, put them into different header files and include them from
there.

-- 
dwmw2


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__ 
  2004-11-30 15:33             ` David Howells
@ 2004-11-30 15:51               ` Linus Torvalds
  2004-11-30 20:47                 ` Alexandre Oliva
  2004-11-30 16:18               ` David Howells
  1 sibling, 1 reply; 185+ messages in thread
From: Linus Torvalds @ 2004-11-30 15:51 UTC (permalink / raw)
  To: David Howells
  Cc: Alexandre Oliva, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, hch, linux-kernel, libc-hacker



On Tue, 30 Nov 2004, David Howells wrote:
> 
> > But in general, I do kind of like the explicit marking. The same way we 
> > explicitly mark the functions inside the kernel that we expose to modules, 
> > we could try to mark the data structures and values that we expose to user 
> > space. That tends to "work".
> 
> That gets trickier when it comes to #defines, I think.
> 
> Do you really object that much to splitting the header files as I proposed?

I object sternuously to your "the header files". If you can't even say 
_which_ header file, I'm not interested.

If you can say "these X header files have this specific problem Y, and if 
we move part Z into a common area A, we'd solve it because B", that's a 
different matter.

See what I'm saying? Whole-sale "move things around because we want to" 
I'm not interested in. Specific problems, yes.

> Personally, I'd prefer us to move to using standard C99 types in lieu of u32
> and co at least for the interface to userspace because they are just that:
> standard.

No. I told you why it cannot and MUST NOT be done. Repeat after me:

	WE MUST NOT POLLUTE THE NAMESPACE!

There are many cases where including a header file means you want the
structure, but it does _not_ mean that you can expose totally unrelated 
types, even if those types happen to be used as part of the structure. See 
what I'm saying?

In contrast, __u8/__u16/__xxx are explicitly defined by the relevant 
standards to be "system name space", and that is _exactly_ why we don't 
use the standard namespaces.

And the fact is, the rules for _when_ you export _which_ standard types
are just too damn hard, and more importantly, they are not something that
the kernel headers should even be deciding. They depend on glibc internal
things like __USE_BSD etc, and they depend on which standard you compile
for, and sometimes even on which _version_ of the standard you compile
for.

I will not have that mess in the kernel. Which means that we do _not_ 
export any of the standard names. Which means that we can't _use_ any of 
them. End of story.

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 15:35               ` David Woodhouse
@ 2004-11-30 15:58                 ` Linus Torvalds
  2004-11-30 16:28                   ` David Woodhouse
  2004-11-30 16:33                 ` David Howells
  1 sibling, 1 reply; 185+ messages in thread
From: Linus Torvalds @ 2004-11-30 15:58 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Alexandre Oliva, Paul Mackerras, Greg KH, Matthew Wilcox,
	David Howells, hch, linux-kernel, libc-hacker



On Tue, 30 Nov 2004, David Woodhouse wrote:
>
> On Mon, 2004-11-29 at 09:41 -0800, Linus Torvalds wrote:
> > In fact, in many ways I'd prefer to have source-level annotations like 
> > "this is exported to user space" over trying to gather things in one 
> > place.
> 
> I don't think you would; not once you really tried it. 
> 
> That's what the littering of ifdef __KERNEL__ attempts to do, and
> there's not really any better way of doing it than that.

The fact is, despite this stupid and way-too-long thread, #ifdef 
__KERNEL__ has worked for over a decade, and works damn well, everything 
considered. 

Remember the second-system-syndrome? It comes from people wanting to fix 
problems in the current implementation, without thinking about all the 
things that it does _well_ (because the things that work are not on their 
radar - they just work). And no, __KERNEL__ may not be pretty, but it's 
damn simple to fix up, and one thing it does really well is allow 
flexibility in a way that forcing structure does not.

I know people like "structure". But it's _waayyy_ overrated. The reason we 
use C instead of some other programming environment is not that C is the 
most highly structured language around, but it's the most _flexible_ one, 
largely because it says "let's give people rope". It allows structure, but 
if you want to cast things around and use gotos, it says "sure, master, 
you're the boss".

Same thing here. The __KERNEL__ approach says "whatever you want, boss".  
It doesn't get in the way. Maybe it doesn't actively _help_ you either,
but you never have to fight any structure it imposes on you.

Also, there _are_ better ways of annotating it than __KERNEL__. In 
particular, if we had a "user annotation", I could make sparse sit up and 
take notice, and _complain_ when you use a non-specific-sized type. That's 
not just theory - Al Viro was talking about that at some point.

			Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__ 
  2004-11-30 15:33             ` David Howells
  2004-11-30 15:51               ` Linus Torvalds
@ 2004-11-30 16:18               ` David Howells
  1 sibling, 0 replies; 185+ messages in thread
From: David Howells @ 2004-11-30 16:18 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Alexandre Oliva, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, hch, linux-kernel


> No. I told you why it cannot and MUST NOT be done. Repeat after me:
> 
> 	WE MUST NOT POLLUTE THE NAMESPACE!

Sorry, I misremembered that bit in your earlier email, but having reread it I
think I have to concur with it. I don't like it very much, but I don't think
there's a good way around it now.

David

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 15:58                 ` Linus Torvalds
@ 2004-11-30 16:28                   ` David Woodhouse
  2004-11-30 16:53                     ` Linus Torvalds
  0 siblings, 1 reply; 185+ messages in thread
From: David Woodhouse @ 2004-11-30 16:28 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Alexandre Oliva, Paul Mackerras, Greg KH, Matthew Wilcox,
	David Howells, hch, linux-kernel, libc-hacker

On Tue, 2004-11-30 at 07:58 -0800, Linus Torvalds wrote:
> The fact is, despite this stupid and way-too-long thread, #ifdef 
> __KERNEL__ has worked for over a decade, and works damn well, everything 
> considered. 

I don't agree with that. The current setup is a complete PITA and that's
why people are clamouring for us to clean the crap up.

But if you feel this strongly about it, then let's go ahead with the
existing mess. We can add more ifdef __KERNEL__ and add rules to the
makefiles to export it for userspace, generating something like the
existing glibc-kernheaders setup. And of course we can look at the
better alternative 'annotations' instead of using #ifdef __KERNEL__.

I think we'll all hate it, and I don't think it's going to really fix
the problem. But we can always reconsider the position later.

> Same thing here. The __KERNEL__ approach says "whatever you want, boss".  
> It doesn't get in the way. Maybe it doesn't actively _help_ you either,
> but you never have to fight any structure it imposes on you.

Having to think before adding something that's user visible is a
_benefit_ not a disadvantage.

-- 
dwmw2


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 15:35               ` David Woodhouse
  2004-11-30 15:58                 ` Linus Torvalds
@ 2004-11-30 16:33                 ` David Howells
  2004-11-30 16:57                   ` Linus Torvalds
  1 sibling, 1 reply; 185+ messages in thread
From: David Howells @ 2004-11-30 16:33 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: David Woodhouse, Alexandre Oliva, Paul Mackerras, Greg KH,
	Matthew Wilcox, hch, linux-kernel, libc-hacker


> The fact is, despite this stupid and way-too-long thread, #ifdef
> __KERNEL__ has worked for over a decade, and works damn well, everything
> considered.

It's a pain, which is why people want to do something about it.

> Remember the second-system-syndrome? It comes from people wanting to fix
> problems in the current implementation, without thinking about all the
> things that it does _well_ (because the things that work are not on their
> radar - they just work). And no, __KERNEL__ may not be pretty, but it's
> damn simple to fix up, and one thing it does really well is allow
> flexibility in a way that forcing structure does not.

And allows abuse and doesn't penalise laziness; both of which do happen in the
kernel header files.

> I know people like "structure". But it's _waayyy_ overrated.

> The reason we use C instead of some other programming environment is not
> that C is the most highly structured language around, but it's the most
> _flexible_ one, largely because it says "let's give people rope".

We use C & ASM for the kernel because you and others won't allow anything
else. Not that I'm necessarily endorsing the use of other languages - I like C
and ASM; I just look at stuff in the kernel occasionally and know that other
languages handle it better and more efficiently. That is, however, besides the
point.

> Same thing here. The __KERNEL__ approach says "whatever you want, boss".
> It doesn't get in the way. Maybe it doesn't actively _help_ you either,
> but you never have to fight any structure it imposes on you.

It can get in the way accidentally.

> Also, there _are_ better ways of annotating it than __KERNEL__. In
> particular, if we had a "user annotation", I could make sparse sit up and
> take notice, and _complain_ when you use a non-specific-sized type. That's
> not just theory - Al Viro was talking about that at some point.

If you did user annotations you'd have to solve the problem of applying it to
#defines and still allowing the constants to be used in assembly. Obviously
this is not impossible. This is trivial with __KERNEL__ or separation into
other files.

David

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 16:28                   ` David Woodhouse
@ 2004-11-30 16:53                     ` Linus Torvalds
  2004-11-30 17:52                       ` David Woodhouse
  2004-11-30 17:55                       ` Al Viro
  0 siblings, 2 replies; 185+ messages in thread
From: Linus Torvalds @ 2004-11-30 16:53 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Alexandre Oliva, Paul Mackerras, Greg KH, Matthew Wilcox,
	David Howells, hch, linux-kernel, libc-hacker



On Tue, 30 Nov 2004, David Woodhouse wrote:
> 
> > Same thing here. The __KERNEL__ approach says "whatever you want, boss".  
> > It doesn't get in the way. Maybe it doesn't actively _help_ you either,
> > but you never have to fight any structure it imposes on you.
> 
> Having to think before adding something that's user visible is a
> _benefit_ not a disadvantage.

I've said this at least three times: if you can point to a _specific_ 
thing you want to move, go wild. I think the big waste in this discussion 
has been that there have _not_ been specific suggestions, just total 
sound-bites like "wouldn't it be great to move things to 'include/kapi'".

If you have a specific thing in mind, say instead something like

	"Wouldn't it be great if we moved all the tty layer IOCTL numbers 
	 into 'tty-ioctl-nr.h', and made the old header file just include 
	 that header file, so that new libc users can get them from just 
	 that header? And btw, here's the patch."

then I might listen. Notice how the only really constructive thing to come 
out of this flame-fest has been a patch by Al that looked perfectly 
reasonable, but that got totally drowned out by the arguing?

Note that even _if_ you have a specific thing in mind, I want to see that 
somebody would say "yes, we'd use that organization". I would not be 
surprised at all if glibc people said that they can't really use any 
re-organization anyway, since they need to support old kernel setups too.

See? Changes that aren't specific enough, or don't actually help things is
what I'm against.

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 16:33                 ` David Howells
@ 2004-11-30 16:57                   ` Linus Torvalds
  0 siblings, 0 replies; 185+ messages in thread
From: Linus Torvalds @ 2004-11-30 16:57 UTC (permalink / raw)
  To: David Howells
  Cc: David Woodhouse, Alexandre Oliva, Paul Mackerras, Greg KH,
	Matthew Wilcox, hch, linux-kernel, libc-hacker



On Tue, 30 Nov 2004, David Howells wrote:
> 
> If you did user annotations you'd have to solve the problem of applying it to
> #defines and still allowing the constants to be used in assembly. Obviously
> this is not impossible. This is trivial with __KERNEL__ or separation into
> other files.

Annotations don't have to be per-entry. It can be something as simple as 
"this particular file is exposed to user space", or "things from here to 
here are exposed to user space".

And while automation is good, sometimes automation isn't even worth it. I 
suspect it might be good just to _comment_ the structures we expose to 
user space, even if it's never used for anything but as a note to tell 
people not to change it.

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 16:53                     ` Linus Torvalds
@ 2004-11-30 17:52                       ` David Woodhouse
  2004-11-30 18:21                         ` Linus Torvalds
  2004-11-30 17:55                       ` Al Viro
  1 sibling, 1 reply; 185+ messages in thread
From: David Woodhouse @ 2004-11-30 17:52 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Alexandre Oliva, Paul Mackerras, Greg KH, Matthew Wilcox,
	David Howells, hch, linux-kernel, libc-hacker

On Tue, 2004-11-30 at 08:53 -0800, Linus Torvalds wrote:
> I've said this at least three times: if you can point to a _specific_ 
> thing you want to move, go wild. I think the big waste in this discussion 
> has been that there have _not_ been specific suggestions, just total 
> sound-bites like "wouldn't it be great to move things to 'include/kapi'".

I've already done some -- see the contents of include/mtd/ for example,
which contain only the parts which userspace should see, while
include/linux/mtd/ is now entirely kernel-private stuff.

We were trying to reach a consensus on where new header files can be put
and indeed whether they should be split at all. We can all disappear for
a week or two and come back with a completely sanitised set of header
files with the userspace stuff in a separate directory and _then_ and be
told "no, we don't like that; do it differently".

A bunch of us have even made a start on such things to prove the
concept. Presenting it as a fait accompli just to have it rejected isn't
much fun though -- the idea was to get some form of agreement in
principle beforehand, then collaborate on actually doing it.

> Note that even _if_ you have a specific thing in mind, I want to see that 
> somebody would say "yes, we'd use that organization". I would not be 
> surprised at all if glibc people said that they can't really use any 
> re-organization anyway, since they need to support old kernel setups too.

The idea in the proposal which David posted, which seemed perfectly
specific enough to me, was to move all the user-visible parts to
separate header files in a separate directory. Those same header files
would also be included by the kernel-private headers. This would be done
in such a way that it's a drop-in replacement for the existing sanitised
glibc-kernheaders package. Of _course_ it needs to be usable without
pain by the glibc people.

We can just go ahead and _do_ that if necessary, but it seemed like a
sane plan to agree on how to do it beforehand.

-- 
dwmw2


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 16:53                     ` Linus Torvalds
  2004-11-30 17:52                       ` David Woodhouse
@ 2004-11-30 17:55                       ` Al Viro
  1 sibling, 0 replies; 185+ messages in thread
From: Al Viro @ 2004-11-30 17:55 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: David Woodhouse, Alexandre Oliva, Paul Mackerras, Greg KH,
	Matthew Wilcox, David Howells, hch, linux-kernel, libc-hacker

On Tue, Nov 30, 2004 at 08:53:34AM -0800, Linus Torvalds wrote:
> then I might listen. Notice how the only really constructive thing to come 
> out of this flame-fest has been a patch by Al that looked perfectly 
> reasonable, but that got totally drowned out by the arguing?

Frankly, I suspect that amount of effort that went into postings made in
that thread is approaching what it would take to clean the damn thing
up for good.  I wasn't kidding about 10 minutes work to do that patch...

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 17:52                       ` David Woodhouse
@ 2004-11-30 18:21                         ` Linus Torvalds
  2004-11-30 20:28                           ` Mariusz Mazur
  2004-11-30 22:48                           ` Matt Mackall
  0 siblings, 2 replies; 185+ messages in thread
From: Linus Torvalds @ 2004-11-30 18:21 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Alexandre Oliva, Paul Mackerras, Greg KH, Matthew Wilcox,
	David Howells, hch, linux-kernel, libc-hacker



On Tue, 30 Nov 2004, David Woodhouse wrote:
> 
> The idea in the proposal which David posted, which seemed perfectly
> specific enough to me, was to move all the user-visible parts to
> separate header files in a separate directory.

You call _that_ specific?

Hell no. You need to do it without breaking existing uses, as noted 
earlier, and it's not specific at all. "all user visible parts" is a big 
undertaking, if you can't make it smaller than that, then forget about it.

Basic rule in kernel engineering: you don't just rewrite the world. You do
it in incremental independent steps.

Any mtd-specific rewrite is obviously a go. 

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 18:21                         ` Linus Torvalds
@ 2004-11-30 20:28                           ` Mariusz Mazur
  2004-11-30 20:47                             ` Linus Torvalds
  2004-11-30 22:48                           ` Matt Mackall
  1 sibling, 1 reply; 185+ messages in thread
From: Mariusz Mazur @ 2004-11-30 20:28 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: David Woodhouse, Alexandre Oliva, Paul Mackerras, Greg KH,
	Matthew Wilcox, David Howells, hch, linux-kernel, libc-hacker

On wtorek 30 listopad 2004 19:21, Linus Torvalds wrote:
> You call _that_ specific?
>
> Hell no. You need to do it without breaking existing uses, as noted
> earlier, and it's not specific at all. "all user visible parts" is a big
> undertaking, if you can't make it smaller than that, then forget about it.
>
> Basic rule in kernel engineering: you don't just rewrite the world. You do
> it in incremental independent steps.
>
> Any mtd-specific rewrite is obviously a go.

Facts:
- current __KERNEL__ stuff is crap; just check any distro what they're using - 
either some kind of userland headers or a patched version of kernel headers
- "You don't use kernel headers in userland!!"
small print: except this, this, and that dir, which are userland friendly
That's quite schizophrenic.

Specific problems:
- glibc has a copy of eg. networking definitions (mostly lots of numbers for 
different protocols); if userland wants to use something that's not in glibc 
already, it has to include linux' headers; which more often than not causes 
conflicts with glibc
- the above problem is present in allmost all headers that have a glibc and 
linux version
- glibc uses linux headers for getting ABI stuff in... dunno... four? five 
cases? And *everything* else ends up getting duplicated.

People in this thread are trying to force you to agree to a specific location 
where stuff like the above mentioned mtd can go to and to start accepting 
patches (afaik there were a number of patches trying to introduce that 
userland dir - all of them ignored). That's (mostly) all.


-- 
In the year eighty five ten
God is gonna shake his mighty head
He'll either say,
"I'm pleased where man has been"
Or tear it down, and start again

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 15:51               ` Linus Torvalds
@ 2004-11-30 20:47                 ` Alexandre Oliva
  2004-11-30 20:56                   ` Linus Torvalds
  0 siblings, 1 reply; 185+ messages in thread
From: Alexandre Oliva @ 2004-11-30 20:47 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: David Howells, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, hch, linux-kernel, libc-hacker

On Nov 30, 2004, Linus Torvalds <torvalds@osdl.org> wrote:

> I object sternuously to your "the header files". If you can't even say 
> _which_ header file, I'm not interested.

How about this formulation then:

- move anything that is not protected by #ifdef __KERNEL__ to the
ukabi header tree, adding an include in the beginning of the original
header that includes the ukabi header.  Since ukabi stuff can't depend on
kernel-internal stuff, having this include as the first thing in the
existing header should work.

- figure out what should have been protected by #ifdef __KERNEL__ but
wasn't, and move it back

- wait for bug reports from the world and deal with them

- repeat until you feel you got it right


> See what I'm saying? Whole-sale "move things around because we want to" 
> I'm not interested in. Specific problems, yes.

The specific problem we're trying to address is the creation of header
files that define the ABI between userland and kernel, that both
kernel and userland can use.

>> Personally, I'd prefer us to move to using standard C99 types in lieu of u32
>> and co at least for the interface to userspace because they are just that:
>> standard.

> No. I told you why it cannot and MUST NOT be done. Repeat after me:

> 	WE MUST NOT POLLUTE THE NAMESPACE!

David, Linus is right.  Using standard types is not an option because
they're not built in type names; you have to include a header to bring
them in, and ideally you shouldn't gratuitously get the names defined
just because you include some unrelated header, although the C
Standard grants standard headers freedom to include other headers or
something along these lines; I don't recall the exact passage of the
Standard that says so, but we should strive to not pollute the user
namespace anyway.

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 20:28                           ` Mariusz Mazur
@ 2004-11-30 20:47                             ` Linus Torvalds
  2004-11-30 22:33                               ` Sam Ravnborg
  2004-12-01  5:50                               ` H. Peter Anvin
  0 siblings, 2 replies; 185+ messages in thread
From: Linus Torvalds @ 2004-11-30 20:47 UTC (permalink / raw)
  To: Mariusz Mazur
  Cc: David Woodhouse, Alexandre Oliva, Paul Mackerras, Greg KH,
	Matthew Wilcox, David Howells, hch, linux-kernel, libc-hacker



On Tue, 30 Nov 2004, Mariusz Mazur wrote:
>
> Facts:
> - current __KERNEL__ stuff is crap;

Fact: you're wrong. I _constantly_ get emails when something breaks. It 
doesn't happen all that often, but it happens.

IOW, I _know_ from personal experience that people depend on it. No ifs, 
buts and maybes about it.

> People in this thread are trying to force you to agree to a specific location 
> where stuff like the above mentioned mtd can go to and to start accepting 
> patches (afaik there were a number of patches trying to introduce that 
> userland dir - all of them ignored). That's (mostly) all.

No. People in this thread have almost uniformly ignored my arguments.

If it's MTD-specific, nobody _cares_ where it goes. Somebody take a 
fricking flying leap of faith here.

If that's all that people want, I hereby proclaim that

	include/asm-xxx/user/xxxx.h
	include/user/xxxx.h

is reserved for user-visible stuff. And now you can send me small and 
localized patches that fix a _particular_ gripe. 

But claiming that __KERNEL__ doesn't work is clearly a bunch of crapola. 
As is the notion that you can somehow do it all. We do it in small pieces.

Happy now?

			Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 20:47                 ` Alexandre Oliva
@ 2004-11-30 20:56                   ` Linus Torvalds
  2004-11-30 21:30                     ` Alexandre Oliva
  0 siblings, 1 reply; 185+ messages in thread
From: Linus Torvalds @ 2004-11-30 20:56 UTC (permalink / raw)
  To: Alexandre Oliva
  Cc: David Howells, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, hch, linux-kernel, libc-hacker



On Tue, 30 Nov 2004, Alexandre Oliva wrote:
> 
> - move anything that is not protected by #ifdef __KERNEL__ to the
> ukabi header tree, adding an include in the beginning of the original
> header that includes the ukabi header.

No. I want stuff that goes into the ABI tree to be clearly _defined_ to be 
user-visible. Not a "let's move it all there and then prune it". 

Leave anything questionable in the current location. And never EVER move 
anything that is kernel-internal to a new "clean" tree, because that would 
be totally pointless. At that point, people would have to edit the "clean" 
tree even for kernel internal stuff. No go.

Also, "ukabi" just isn't going to fly as a name. It's also not as simple 
as you seem to think, since a lot of these ABI things are architecture- 
dependent, which apparently all you guys have totally ignored. 

I've suggested "include/user/" and "include/asm-xxx/user", which handles 
architecture-specific parts too. I'm ok with doing it the other way 
around, ie "include/user/" and "include/user/arch-xxxx".

And "user" might be "user-abi" or something like that, but it sure isn't 
going to be some unreadable contraction.

And I _still_ want to see these patches only for things where somebody can 
validly argue that 
 (a) it can't break anything (ie the old location still includes the new 
     one, exactly the same way)
 (b) there are people who will actually take _advantage_ of that 
     particular file (ie "just because I think so" doesn't fly).

Moving files around is just too disruptive to be done without damn good 
reason.

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29 23:00                 ` Linus Torvalds
                                     ` (2 preceding siblings ...)
  2004-11-30  6:44                   ` bert hubert
@ 2004-11-30 21:22                   ` Alexandre Oliva
  2004-11-30 22:22                     ` Linus Torvalds
  3 siblings, 1 reply; 185+ messages in thread
From: Alexandre Oliva @ 2004-11-30 21:22 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox,
	David Howells, hch, linux-kernel, libc-hacker

On Nov 29, 2004, Linus Torvalds <torvalds@osdl.org> wrote:

> On Mon, 29 Nov 2004, Alexandre Oliva wrote:
>> 
>> I don't see it as obvious at all.  The need for an agreement between
>> two parties on an ABI doesn't imply that one party gets to define it
>> and the other gets to follow it. 

> Sorry, but that's not how it works.

Then maybe this is the fundamental problem.  As long as the kernel
doesn't recognize that an ABI is a contract, rather than an
imposition, kernel developers won't care.

But guess what?  Kernel developers *do* care about maintaining the
userland ABI.  They don't just go wild changing data types,
structures, renumbering system calls and all that sort of nasty stuff
because there's a contract with userland that they don't want to
break.  Because breaking this contract would be a stupid thing, that
would require all affected userland to be modified to cope with the
change, even if that's just a recompile using the new kernel
headers...  erhm...  using the result of applying the kernel ABI
change in the copy/extract/whatever of kernel headers that userland is
entitled to use.


As long as we recognize that the notion that the kernel<->userland ABI
is a separate entity, that enables kernel and userland to exchange
information, rather than something that the kernel defines and
userland must follow, we give strength to the argument that userland
is not a derived work from the kernel.  If this argument proves to be
false, then only works released under the GNU GPL or a compatible
license will be allowed to run atop of Linux, because libc would be a
derived work, its license would become full GPL, and any program
linked with it could only be distributed under the full GPL.  Oh, and
just in case, IANAL :-)

Again, I wouldn't mind that at all, but I'm sure others feel
otherwise.  Besides, as I wrote before, if this is the intent, I'd
rather have it exposed in clear, rather than seeing people being
misled to believe it's fair game to run non-GPLed software on
Linux-based operating systems.

> He who writes the code decides what it is.

I've been in projects in which I wrote ABI headers before the
corresponding kernel port had even started.  Sometimes, quite
unfortunately, the kernel engineer went ahead and wrote headers with
slightly different assumptions, and then we had to get together to
define what the ABI was going to be.

In fact, the way you seem to want things to be, you want both kernel
and userland to write the code, and somehow hope they magically match
each other.  That's not how things work.  The way things work is that
either you get agreement first, or each end goes on its way and then
at some point they figure out where they diverged and fix things up,
or one end goes first and the other takes the definitions and uses
them.  It's in no way a kernel->userland one-way street.  It's a
language (for lack of a better word) that kernel and userland choose
to exchange information, not one that the kernel invents and then
requires others interested in talking to it to learn (you know, that
``Do you speak English/my language?´´ phrase so common in Hollywood
movies in which an American character needs to talk to someone abroad
:-)

Sometimes you'll even find formal ABI specifications, written down
before either userland or kernel start being developed.  Claiming that
the kernel gets to set the ABI as it wishes, without worrying about
userland because it never calls userland is a very narrow view, and in
some cases it's even a lie.  Think signal handling and process
start-up, for example: if you don't follow the userland ABI to call
the function pointer given to the signal function, or if you don't
follow the userland ABI to start the program's entry point, it's not
userland that is broken, it's the kernel.

> In this case, if the kernel 
> does a new extension, it's the kernel that gets to decide what it is. 

As in, say, NPTL?  As if a number of userland considerations hadn't
driven most of the design decisions behind the then-defined ABI
extensions?

> If glibc wants to do something new, go wild. The kernel won't care.

This is not about glibc.  That's not the only userland component.
There are several different alternatives one can use.

And then, the kernel will care, because it has to.  A kernel without
userland is nothing.  A kernel that kept introducing incompatible ABI
changes would die a very quick death.  A kernel has to be very careful
about complying with the ABI it shares with userland, and you know
that, because you do care about it.

> And that's really the fundamental issue. The kernel does not care what
> user land does. The kernel exports functionality, the kernel does _not_
> ask user land to help.

I'm not so much talking about the hundreds of syscalls, the syscall
conventions and the number and type of arguments given to each
syscall.  This is not even well defined in kernel headers.  All that's
exported regarding syscalls is a mapping from syscall names to
numbers.

What I'm talking about is the data structures.  Sure, the kernel can
define these however it likes.  So can userland.  The point is that,
if they don't match, when you get userland and kernel together, in the
hopes of getting something useful done, it doesn't work.

Both ends can come up with extensions, and do things the old way until
the other end picks up the extension.  I'll give you that many
extensions make to the kernel before say glibc, but that's mostly just
like, when we create a new toolchain port, assembler and linker are
generally implemented before the compiler.  I guess it just makes more
sense.  It wastes less effort to make sure you got something right in
a lower layer before you build something atop of it.  It's unlikely
that a change would make to glibc if it used a feature that isn't
available in the kernel, just because most often features undergo
(minor or sometimes major) changes when they're integrated, and it's
easier to fix things up if you haven't committed to the way that
didn't make it there yet.

> That _does_ make it a one-way street. Sorry.

Good, we're getting somewhere.  This is surely better than a zero-way
street, i.e., one in which headers that specify the ABI don't flow in
any direction.  If we're agreeing to make it one way, let's make it
so: have headers that are maintained as part of the kernel, and that
are imposed, if you like, on userland.  Saying it's a one-way street
but denying others from taking what comes their way through this
one-way street hasn't been very effective :-)

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 20:56                   ` Linus Torvalds
@ 2004-11-30 21:30                     ` Alexandre Oliva
  2004-11-30 21:39                       ` Alexandre Oliva
  2004-11-30 22:25                       ` Linus Torvalds
  0 siblings, 2 replies; 185+ messages in thread
From: Alexandre Oliva @ 2004-11-30 21:30 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: David Howells, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, hch, linux-kernel, libc-hacker

On Nov 30, 2004, Linus Torvalds <torvalds@osdl.org> wrote:

> On Tue, 30 Nov 2004, Alexandre Oliva wrote:
>> 
>> - move anything that is not protected by #ifdef __KERNEL__ to the
>> ukabi header tree, adding an include in the beginning of the original
>> header that includes the ukabi header.

> No. I want stuff that goes into the ABI tree to be clearly _defined_ to be 
> user-visible. Not a "let's move it all there and then prune it". 

Right.  Which brings us to my second bullet.  I *knew* I should have
made it the first :-/

> Also, "ukabi" just isn't going to fly as a name. It's also not as simple 
> as you seem to think, since a lot of these ABI things are architecture- 
> dependent, which apparently all you guys have totally ignored. 

Looks like you missed the post that started this thread.  It *did*
mention machine-independent and machine-specific user headers.

> I've suggested "include/user/" and "include/asm-xxx/user", which handles 
> architecture-specific parts too. I'm ok with doing it the other way 
> around, ie "include/user/" and "include/user/arch-xxxx".

As I pointed out, `user' is a very bad name.  As you said yourself,
we're talking about the *kernel* ABI.  So what's `user' supposed to
mean?  Was I so successful in my arguments that you now see it as the
userland ABI? :-)

>  (a) it can't break anything (ie the old location still includes the new 
>      one, exactly the same way)

You mean it can't break anything in a kernel build, or it can't break
anything except for userland apps that abused kernel headers and used
to get away with that?

>  (b) there are people who will actually take _advantage_ of that 
>      particular file (ie "just because I think so" doesn't fly).

People who currently get to maintain duplicates of these header
contents will take immediate advantage of these changes, since they
will no longer have to maintain the duplicates.  This is a major step
forward in terms of maintainability for everybody else, at no expense
to the kernel, that now gets to explicitly (as opposed to implicitly)
document what is the ABI it's willing to comply with.  It's a win for
both ends.

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 21:30                     ` Alexandre Oliva
@ 2004-11-30 21:39                       ` Alexandre Oliva
  2004-11-30 23:33                         ` Matthew Wilcox
  2004-11-30 22:25                       ` Linus Torvalds
  1 sibling, 1 reply; 185+ messages in thread
From: Alexandre Oliva @ 2004-11-30 21:39 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: David Howells, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, hch, linux-kernel, libc-hacker

On Nov 30, 2004, Alexandre Oliva <aoliva@redhat.com> wrote:

>> I've suggested "include/user/" and "include/asm-xxx/user", which handles 
>> architecture-specific parts too. I'm ok with doing it the other way 
>> around, ie "include/user/" and "include/user/arch-xxxx".

> As I pointed out, `user' is a very bad name.  As you said yourself,
> we're talking about the *kernel* ABI.  So what's `user' supposed to
> mean?  Was I so successful in my arguments that you now see it as the
> userland ABI? :-)

I got carried away with joking and failed to repeat why I consider it
a bad name (assuming that, since you missed the beginning of the
thread, you probably missed the first reply I posted to the message
that started it): since we're going to install these headers in
/usr/include (where headers for userland live), /usr/include/user is
quite misleading.  /usr/include/kernel would be far more
appropriate for this purpose.

Sure, we could take headers from linux-*/include/user and install them
in /usr/include/kernel, but then includes in there that reference
other headers in user/ or in asm-<arch>/ will cease to work.

So we should come up with a name that makes sense for both users of
these headers, which is why I suggested ukabi.  linux/abi and
asm-<mach>/abi work just as well, and then we can soft-link `abi -> .'
in /usr/include/{linux,asm-<mach>} if needed.  Ideally, we wouldn't
have to.

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 21:22                   ` Alexandre Oliva
@ 2004-11-30 22:22                     ` Linus Torvalds
  2004-11-30 22:57                       ` Alexandre Oliva
  0 siblings, 1 reply; 185+ messages in thread
From: Linus Torvalds @ 2004-11-30 22:22 UTC (permalink / raw)
  To: Alexandre Oliva
  Cc: Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox,
	David Howells, hch, linux-kernel, libc-hacker



On Tue, 30 Nov 2004, Alexandre Oliva wrote:
> 
> Then maybe this is the fundamental problem.  As long as the kernel
> doesn't recognize that an ABI is a contract, rather than an
> imposition, kernel developers won't care.

That's a silly analogy. Worse, it's a very flawed analogy.

If you want to use a legal analogy, the ABI is not a contract, it's a
public _license_.

Why? A contract is something that you cannot change unilaterally. Clearly
the kernel _can_ and will change the ABI without asking every existing
program for permission.

In a license, you can always just _expand_ the license eventually. Maybe
you originally licensed for "fly-fishing for trout", and later you can
expand that license to say "you can also catch crawfish" without impacting
your existing licensees.

And exactly as with an ABI, the only thing you can't do is _remove_ rights 
without getting into trouble. 

So get your analogies straight. The kernel ABI is _not_ a contract. 

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 21:30                     ` Alexandre Oliva
  2004-11-30 21:39                       ` Alexandre Oliva
@ 2004-11-30 22:25                       ` Linus Torvalds
  2004-11-30 22:34                         ` David Woodhouse
                                           ` (2 more replies)
  1 sibling, 3 replies; 185+ messages in thread
From: Linus Torvalds @ 2004-11-30 22:25 UTC (permalink / raw)
  To: Alexandre Oliva
  Cc: David Howells, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, hch, linux-kernel, libc-hacker



On Tue, 30 Nov 2004, Alexandre Oliva wrote:

> >  (a) it can't break anything (ie the old location still includes the new 
> >      one, exactly the same way)
> 
> You mean it can't break anything in a kernel build, or it can't break
> anything except for userland apps that abused kernel headers and used
> to get away with that?

It can't break userland either.

> >  (b) there are people who will actually take _advantage_ of that 
> >      particular file (ie "just because I think so" doesn't fly).
> 
> People who currently get to maintain duplicates of these header
> contents will take immediate advantage of these changes, since they
> will no longer have to maintain the duplicates.

Wrong. They'll _still_ have to maintain duplicates, since they can't rely
ont he end-user to have a recent enough kernel.

It's just that they can hopefully start _copying_ their dupliates more 
easily. But if you think the duplication goes away, then I don't want you 
to send me patches, because you don't understand the issues.

At RH you may see only the case where people do a whole-system upgrade. 
Please realize that that is just _one_ schenario, and you should not 
assume that it's the only valid one.

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 20:47                             ` Linus Torvalds
@ 2004-11-30 22:33                               ` Sam Ravnborg
  2004-11-30 22:44                                 ` Mariusz Mazur
  2004-12-01  5:50                               ` H. Peter Anvin
  1 sibling, 1 reply; 185+ messages in thread
From: Sam Ravnborg @ 2004-11-30 22:33 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Mariusz Mazur, David Woodhouse, Alexandre Oliva, Paul Mackerras,
	Greg KH, Matthew Wilcox, David Howells, hch, linux-kernel,
	libc-hacker

On Tue, Nov 30, 2004 at 12:47:50PM -0800, Linus Torvalds wrote:
 If that's all that people want, I hereby proclaim that
> 
> 	include/asm-xxx/user/xxxx.h
> 	include/user/xxxx.h
> 
> is reserved for user-visible stuff. And now you can send me small and 
> localized patches that fix a _particular_ gripe. 

Please use:
 	include/$arch/user-asm/xxxx.h
 	include/user/xxxx.h
	
This allows us to

1) To include file foo.h we still distingush between user versus user-asm:
#include <user/foo.h>		(include/user/foo.h)
#include <user-asm/foo.h>	(include/$arch/user-asm/foo.h)

2) No symlinks needed - just proper options to gcc

	Sam

This should do it for the top-level makefile:

===== Makefile 1.549 vs edited =====
--- 1.549/Makefile	2004-11-15 10:00:11 +01:00
+++ edited/Makefile	2004-11-30 23:31:09 +01:00
@@ -345,6 +345,10 @@
 # Needed to be compatible with the O= option
 LINUXINCLUDE    := -Iinclude \
                    $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include)
+		   
+# Extend include path with user dir
+LINUXINCLUDE	+= -Iinclude/$(ARCH) \
+		   $(if $(KBUILD_SRC), -I$(srctree)/include/$(ARCH))
 
 CPPFLAGS        := -D__KERNEL__ $(LINUXINCLUDE)
 


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 22:25                       ` Linus Torvalds
@ 2004-11-30 22:34                         ` David Woodhouse
  2004-11-30 22:51                           ` Linus Torvalds
  2004-11-30 22:52                         ` [RFC] Splitting kernel headers and deprecating __KERNEL__ Mariusz Mazur
  2004-11-30 23:05                         ` Alexandre Oliva
  2 siblings, 1 reply; 185+ messages in thread
From: David Woodhouse @ 2004-11-30 22:34 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Alexandre Oliva, dhowells, Paul Mackerras, Greg KH,
	Matthew Wilcox, hch, linux-kernel, libc-hacker

On Tue, 2004-11-30 at 14:25 -0800, Linus Torvalds wrote:
> 
> On Tue, 30 Nov 2004, Alexandre Oliva wrote:
> 
> > >  (a) it can't break anything (ie the old location still includes the new 
> > >      one, exactly the same way)
> > 
> > You mean it can't break anything in a kernel build, or it can't break
> > anything except for userland apps that abused kernel headers and used
> > to get away with that?
> 
> It can't break userland either.

That depends on your definition of 'break'. It should prevent abuse.

To pick a specific example, since you like them: where userland programs
are including atomic.h, and hence writing programs which don't compile
on some architectures, and which compile on others but silently give
non-atomic results, it's perfectly acceptable and indeed advisable to
prevent compilation across the board.

Some people might call that breakage; I don't.

-- 
dwmw2


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 22:33                               ` Sam Ravnborg
@ 2004-11-30 22:44                                 ` Mariusz Mazur
  2004-11-30 23:03                                   ` Al Viro
  0 siblings, 1 reply; 185+ messages in thread
From: Mariusz Mazur @ 2004-11-30 22:44 UTC (permalink / raw)
  To: Sam Ravnborg
  Cc: Linus Torvalds, David Woodhouse, Alexandre Oliva, Paul Mackerras,
	Greg KH, Matthew Wilcox, David Howells, hch, linux-kernel,
	libc-hacker

On wtorek 30 listopad 2004 23:33, Sam Ravnborg wrote:
> On Tue, Nov 30, 2004 at 12:47:50PM -0800, Linus Torvalds wrote:
>  If that's all that people want, I hereby proclaim that
>
> >  include/asm-xxx/user/xxxx.h
> >  include/user/xxxx.h
> >
> > is reserved for user-visible stuff. And now you can send me small and
> > localized patches that fix a _particular_ gripe.
>
> Please use:
>   include/$arch/user-asm/xxxx.h
>   include/user/xxxx.h

Wrong. These dirs must be linked to /usr/include so they must have more 
meaningfull names.

-- 
In the year eighty five ten
God is gonna shake his mighty head
He'll either say,
"I'm pleased where man has been"
Or tear it down, and start again

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 18:21                         ` Linus Torvalds
  2004-11-30 20:28                           ` Mariusz Mazur
@ 2004-11-30 22:48                           ` Matt Mackall
  2004-11-30 22:51                             ` Christoph Hellwig
  2004-11-30 22:55                             ` Linus Torvalds
  1 sibling, 2 replies; 185+ messages in thread
From: Matt Mackall @ 2004-11-30 22:48 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: David Woodhouse, Alexandre Oliva, Paul Mackerras, Greg KH,
	Matthew Wilcox, David Howells, hch, linux-kernel, libc-hacker

On Tue, Nov 30, 2004 at 10:21:52AM -0800, Linus Torvalds wrote:
> 
> 
> On Tue, 30 Nov 2004, David Woodhouse wrote:
> > 
> > The idea in the proposal which David posted, which seemed perfectly
> > specific enough to me, was to move all the user-visible parts to
> > separate header files in a separate directory.
> 
> You call _that_ specific?
> 
> Hell no. You need to do it without breaking existing uses, as noted 
> earlier, and it's not specific at all. "all user visible parts" is a big 
> undertaking, if you can't make it smaller than that, then forget about it.

So we follow dhowell's plan with the following additions:

a) during the transition, include/linux/foo.h includes
include/user/foo.h if it exists and similarly for asm-*
b) when include/user is deemed sufficiently populated, a flag day is
declared and links from /usr/include are switched to them
c) #define __KERNEL__ is dropped from the compile
d) extraneous include/user/* includes are removed from include/linux
(though they'll often make perfect sense)

Step b) may be a long time in coming, but for interfaces where kernel
headers are frequently abused, we can start feeling benefits.

> Basic rule in kernel engineering: you don't just rewrite the world. You do
> it in incremental independent steps.

Even so, it's sometimes important for everyone to agree on a
destination before they set out..

--
Mathematics is the supreme nostalgia of our time.

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 22:34                         ` David Woodhouse
@ 2004-11-30 22:51                           ` Linus Torvalds
  2004-11-30 23:50                             ` David Woodhouse
  0 siblings, 1 reply; 185+ messages in thread
From: Linus Torvalds @ 2004-11-30 22:51 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Alexandre Oliva, dhowells, Paul Mackerras, Greg KH,
	Matthew Wilcox, hch, linux-kernel, libc-hacker



On Tue, 30 Nov 2004, David Woodhouse wrote:
> 
> That depends on your definition of 'break'. It should prevent abuse.

Not really. 

It should prevent _future_ abuse.

The notion of "preventing existing xxx" is insane. You can't "prevent" 
something that already happened unless you've come up with some new 
interesting theory of causality.

> To pick a specific example, since you like them: where userland programs
> are including atomic.h, and hence writing programs which don't compile
> on some architectures, and which compile on others but silently give
> non-atomic results, it's perfectly acceptable and indeed advisable to
> prevent compilation across the board.
> 
> Some people might call that breakage; I don't.

I do. The thing is, the people who _notice_ the breakage are often the 
people who don't know what the hell to do about it.

The way to prevent _future_ abuse is by adding something like

	#ifndef __KERNEL__
	#warning "This really doesn't work"
	#endif

which does that, and has the advantage of not breaking anything at all.

In other words: if you want to move things around just to break things, 
THEN THAT IS INCREDIBLY STUPID. We don't do things to screw our users 
over. 

Feel free to send a patch.

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 22:48                           ` Matt Mackall
@ 2004-11-30 22:51                             ` Christoph Hellwig
  2004-11-30 23:09                               ` Chris Friesen
  2004-11-30 22:55                             ` Linus Torvalds
  1 sibling, 1 reply; 185+ messages in thread
From: Christoph Hellwig @ 2004-11-30 22:51 UTC (permalink / raw)
  To: Matt Mackall
  Cc: Linus Torvalds, David Woodhouse, Alexandre Oliva, Paul Mackerras,
	Greg KH, Matthew Wilcox, David Howells, hch, linux-kernel,
	libc-hacker

> a) during the transition, include/linux/foo.h includes
> include/user/foo.h if it exists and similarly for asm-*
> b) when include/user is deemed sufficiently populated, a flag day is
> declared and links from /usr/include are switched to them

there are no such links, only copies (more or less modified)


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 22:25                       ` Linus Torvalds
  2004-11-30 22:34                         ` David Woodhouse
@ 2004-11-30 22:52                         ` Mariusz Mazur
  2004-11-30 23:05                           ` Linus Torvalds
  2004-11-30 23:05                         ` Alexandre Oliva
  2 siblings, 1 reply; 185+ messages in thread
From: Mariusz Mazur @ 2004-11-30 22:52 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Alexandre Oliva, David Howells, Paul Mackerras, Greg KH,
	David Woodhouse, Matthew Wilcox, hch, linux-kernel, libc-hacker

On wtorek 30 listopad 2004 23:25, Linus Torvalds wrote:
> > You mean it can't break anything in a kernel build, or it can't break
> > anything except for userland apps that abused kernel headers and used
> > to get away with that?
>
> It can't break userland either.

Userland? And who's using vanilla kernel headers in userland? Duplicates 
maintainers can always fix stuff independent of kernel (and I do).

> > >  (b) there are people who will actually take _advantage_ of that
> > >      particular file (ie "just because I think so" doesn't fly).
> >
> > People who currently get to maintain duplicates of these header
> > contents will take immediate advantage of these changes, since they
> > will no longer have to maintain the duplicates.
>
> Wrong. They'll _still_ have to maintain duplicates, since they can't rely
> ont he end-user to have a recent enough kernel.

The end user uses a distro that does these things for him. If not, he needs to 
know his way around with finding working kernel headers, otherwise he won't 
be able to build anything (he surely isn't able now - which btw you're trying 
really hard to ignore).
I don't get your point.

> It's just that they can hopefully start _copying_ their dupliates more
> easily. But if you think the duplication goes away, then I don't want you
> to send me patches, because you don't understand the issues.

Yes. The more stuff gets separated, the less work duplicates maintainers have. 
Once all userland stuff gets extracted, those maintainers won't have anything 
else to do, maybe except releasing their final version and using it for 
backward compatibility. And that's what we're aiming at.


-- 
In the year eighty five ten
God is gonna shake his mighty head
He'll either say,
"I'm pleased where man has been"
Or tear it down, and start again

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 22:48                           ` Matt Mackall
  2004-11-30 22:51                             ` Christoph Hellwig
@ 2004-11-30 22:55                             ` Linus Torvalds
  2004-11-30 23:29                               ` Matt Mackall
  2004-11-30 23:51                               ` David Woodhouse
  1 sibling, 2 replies; 185+ messages in thread
From: Linus Torvalds @ 2004-11-30 22:55 UTC (permalink / raw)
  To: Matt Mackall
  Cc: David Woodhouse, Alexandre Oliva, Paul Mackerras, Greg KH,
	Matthew Wilcox, David Howells, hch, linux-kernel, libc-hacker



On Tue, 30 Nov 2004, Matt Mackall wrote:
> 
> So we follow dhowell's plan with the following additions:

No.

We do _not_ move stuff over that is questionable.

I thought that was clear by now. The rules are:
 - we only move things that _have_ to move
 - we don't break existing programs, and no "but they are broken already" 
   is not an excuse.
 - we only move things where that _particular_ move can be shown to be 
   beneficial.

No whole-sale moves. No "let's break things that I think are broken". No 
"let's change things because we can".

Well-defined moves. Both in content _and_ in reason.

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 22:22                     ` Linus Torvalds
@ 2004-11-30 22:57                       ` Alexandre Oliva
  2004-11-30 23:12                         ` Linus Torvalds
  0 siblings, 1 reply; 185+ messages in thread
From: Alexandre Oliva @ 2004-11-30 22:57 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox,
	David Howells, hch, linux-kernel, libc-hacker

On Nov 30, 2004, Linus Torvalds <torvalds@osdl.org> wrote:

> On Tue, 30 Nov 2004, Alexandre Oliva wrote:

>> Then maybe this is the fundamental problem.  As long as the kernel
>> doesn't recognize that an ABI is a contract, rather than an
>> imposition, kernel developers won't care.

> That's a silly analogy. Worse, it's a very flawed analogy.

> If you want to use a legal analogy, the ABI is not a contract, it's a
> public _license_.

I didn't mean to use a legal analogy.  I meant contract in the
software engineering sense.  Sorry if that wasn't clear.

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 22:44                                 ` Mariusz Mazur
@ 2004-11-30 23:03                                   ` Al Viro
  2004-11-30 23:08                                     ` Mariusz Mazur
                                                       ` (2 more replies)
  0 siblings, 3 replies; 185+ messages in thread
From: Al Viro @ 2004-11-30 23:03 UTC (permalink / raw)
  To: Mariusz Mazur
  Cc: Sam Ravnborg, Linus Torvalds, David Woodhouse, Alexandre Oliva,
	Paul Mackerras, Greg KH, Matthew Wilcox, David Howells, hch,
	linux-kernel, libc-hacker

On Tue, Nov 30, 2004 at 11:44:21PM +0100, Mariusz Mazur wrote:
> On wtorek 30 listopad 2004 23:33, Sam Ravnborg wrote:
> > On Tue, Nov 30, 2004 at 12:47:50PM -0800, Linus Torvalds wrote:
> >  If that's all that people want, I hereby proclaim that
> >
> > >  include/asm-xxx/user/xxxx.h
> > >  include/user/xxxx.h
> > >
> > > is reserved for user-visible stuff. And now you can send me small and
> > > localized patches that fix a _particular_ gripe.
> >
> > Please use:
> >   include/$arch/user-asm/xxxx.h
> >   include/user/xxxx.h
> 
> Wrong. These dirs must be linked to /usr/include so they must have more 
> meaningfull names.

WTF?  I've got a dozen kernel trees hanging around, which one (and WTF any,
while we are at it) should be "linked to"?

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 22:25                       ` Linus Torvalds
  2004-11-30 22:34                         ` David Woodhouse
  2004-11-30 22:52                         ` [RFC] Splitting kernel headers and deprecating __KERNEL__ Mariusz Mazur
@ 2004-11-30 23:05                         ` Alexandre Oliva
  2 siblings, 0 replies; 185+ messages in thread
From: Alexandre Oliva @ 2004-11-30 23:05 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: David Howells, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, hch, linux-kernel, libc-hacker

On Nov 30, 2004, Linus Torvalds <torvalds@osdl.org> wrote:

> Wrong. They'll _still_ have to maintain duplicates, since they can't rely
> ont he end-user to have a recent enough kernel.

No.  They just won't have the headers for the newer kernel installed
on their system.  They stick with the kernel ABI headers provided by a
baseline kernel, and that's probably from an older kernel tarball.

> It's just that they can hopefully start _copying_ their dupliates more 
> easily.

It's still the headers from the kernel, copied or not.  From a
different, baseline version, maybe.

> At RH you may see only the case where people do a whole-system upgrade. 

I personally don't care about the case you perceive as the Red Hat
case, because I work on embedded systems development, in a different
organization within Red Hat than the people who develop Red Hat
Enterprise Linux and Fedora.  So yes, in the case that actually
benefits me, it's even more of a whole-system thing, since for our
group it's *the* kernel, not *a* random kernel version.  It's the
kernel we port and ship to the customer, and the corresponding
kernel<->userland ABI.

That said, I do understand the issues of kernel baseline ABI settings,
or at least part of them.  I know you don't want to assume the
presence of a certain system call just because you're running a kernel
that implements it.  In fact, I know you don't want to assume the
presence of a system call just because it's defined in linux/unistd.h.
As long as the userland software keeps this distinction in mind, I
don't see why taking the latest kernel<->userland ABI headers, that
describe the newest system calls and any corresponding data
structures, could hurt anything.  In fact, if you don't have the
corresponding system call numbers and data structures, you can't
possibly take advantage of the new syscalls even if they happen to be
available, unless you duplicate the syscall numbers and data structure
definitions in your own code, which, from a software engineering
standpoint, is bad.

Of course, badly written software can assume the syscalls are present
just because their numbers are given, and then people who rely on such
bad software lose, but such is the nature of bad software.

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 22:52                         ` [RFC] Splitting kernel headers and deprecating __KERNEL__ Mariusz Mazur
@ 2004-11-30 23:05                           ` Linus Torvalds
  0 siblings, 0 replies; 185+ messages in thread
From: Linus Torvalds @ 2004-11-30 23:05 UTC (permalink / raw)
  To: Mariusz Mazur
  Cc: Alexandre Oliva, David Howells, Paul Mackerras, Greg KH,
	David Woodhouse, Matthew Wilcox, hch, linux-kernel, libc-hacker



On Tue, 30 Nov 2004, Mariusz Mazur wrote:
> 
> Userland? And who's using vanilla kernel headers in userland? Duplicates 
> maintainers can always fix stuff independent of kernel (and I do).

Dammit, how hard is it for people to admit that they aren't the only ones 
around?

There are still people who use ancient setups, and upgrade the kernel. 
They won't complain to _you_ or to redhat when their setup breaks, because 
they aren't _using_ your setup. They complain to me.

This is what statisticians call "self-selection". It means that you guys 
only see the part of the world that uses your stuff, so you think that 
there is nothing else. 

> I don't get your point.

Very obviously. 

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 23:03                                   ` Al Viro
@ 2004-11-30 23:08                                     ` Mariusz Mazur
  2004-12-01  5:23                                       ` Sam Ravnborg
  2004-11-30 23:13                                     ` Alexandre Oliva
  2004-11-30 23:36                                     ` Chris Friesen
  2 siblings, 1 reply; 185+ messages in thread
From: Mariusz Mazur @ 2004-11-30 23:08 UTC (permalink / raw)
  To: Al Viro
  Cc: Sam Ravnborg, Linus Torvalds, David Woodhouse, Alexandre Oliva,
	Paul Mackerras, Greg KH, Matthew Wilcox, David Howells, hch,
	linux-kernel, libc-hacker

On środa 01 grudzień 2004 00:03, Al Viro wrote:
> > Wrong. These dirs must be linked to /usr/include so they must have more
> > meaningfull names.
>
> WTF?  I've got a dozen kernel trees hanging around, which one (and WTF any,
> while we are at it) should be "linked to"?

Linked, copied, mount --binded, whatever. Just not under the 
name /usr/include/user, but something more meaningfull.

And to directly answer your question: you'll 'link' the newest kernel you've 
got around (we all know linux doesn't break abi compatibility, so the newest 
kernel will be fully compatible with all other kernel lying around, right? :)

-- 
In the year eighty five ten
God is gonna shake his mighty head
He'll either say,
"I'm pleased where man has been"
Or tear it down, and start again

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 22:51                             ` Christoph Hellwig
@ 2004-11-30 23:09                               ` Chris Friesen
  2004-11-30 23:35                                 ` Matt Mackall
  2004-12-01  0:23                                 ` Miquel van Smoorenburg
  0 siblings, 2 replies; 185+ messages in thread
From: Chris Friesen @ 2004-11-30 23:09 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: Matt Mackall, Linus Torvalds, David Woodhouse, Alexandre Oliva,
	Paul Mackerras, Greg KH, Matthew Wilcox, David Howells,
	linux-kernel, libc-hacker

Christoph Hellwig wrote:

>>b) when include/user is deemed sufficiently populated, a flag day is
>>declared and links from /usr/include are switched to them
> 
> 
> there are no such links, only copies (more or less modified)

This may be somewhat heretical, but someone has to ask...

Once include/user/foo.h is sufficiently clean and sufficiently complete, is 
there any reason to not allow such links?

Chris

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 22:57                       ` Alexandre Oliva
@ 2004-11-30 23:12                         ` Linus Torvalds
  2004-12-01 19:41                           ` Alexandre Oliva
  0 siblings, 1 reply; 185+ messages in thread
From: Linus Torvalds @ 2004-11-30 23:12 UTC (permalink / raw)
  To: Alexandre Oliva
  Cc: Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox,
	David Howells, hch, linux-kernel, libc-hacker



On Tue, 30 Nov 2004, Alexandre Oliva wrote:

> On Nov 30, 2004, Linus Torvalds <torvalds@osdl.org> wrote:
> 
> > On Tue, 30 Nov 2004, Alexandre Oliva wrote:
> 
> >> Then maybe this is the fundamental problem.  As long as the kernel
> >> doesn't recognize that an ABI is a contract, rather than an
> >> imposition, kernel developers won't care.
> 
> > That's a silly analogy. Worse, it's a very flawed analogy.
> 
> > If you want to use a legal analogy, the ABI is not a contract, it's a
> > public _license_.
> 
> I didn't mean to use a legal analogy.  I meant contract in the
> software engineering sense.  Sorry if that wasn't clear.

Then your definition of a "contract" is flawed or your world-view has 
nothing to do with reality.

An ABI is not a contract. It's somebody exposing a certain set of 
features. It does _not_ mean that the party that gets exposed has any say 
wrt the features. At most he can hope that the exported set doesn't 
shrink or change semantically, but let's face it, even that does happen.

The Linux kernel has been very very good about this exposure, in the sense 
that we try very very hard not to break it. I think that's a good thing. 
But that in no way makes anything a "contract".

It's the same thing with CPU vendors. You as a programmer, didn't get to 
dictate what the CPU vendor implemented. You take it or leave it, and at 
most you can make suggestions about things that might make the CPU vendor 
sell more chips. The ISA is not a "contract" - it's just an interface that 
was handed down to you. You didn't get to negotiate, you got to say "yes" 
or "no".

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 23:03                                   ` Al Viro
  2004-11-30 23:08                                     ` Mariusz Mazur
@ 2004-11-30 23:13                                     ` Alexandre Oliva
  2004-11-30 23:36                                     ` Chris Friesen
  2 siblings, 0 replies; 185+ messages in thread
From: Alexandre Oliva @ 2004-11-30 23:13 UTC (permalink / raw)
  To: Al Viro
  Cc: Mariusz Mazur, Sam Ravnborg, Linus Torvalds, David Woodhouse,
	Paul Mackerras, Greg KH, Matthew Wilcox, David Howells, hch,
	linux-kernel, libc-hacker

On Nov 30, 2004, Al Viro <viro@parcelfarce.linux.theplanet.co.uk> wrote:

> WTF?  I've got a dozen kernel trees hanging around, which one (and WTF any,
> while we are at it) should be "linked to"?

Whichever you chose to install in your /usr/include, and use as the
kernel ABI definition as far as userland is concerned.

I don't think `make install' should touch /usr/include at all.  It
should be a separate step, such that one can build a kernel abi
headers package out of the kernel source tree, ideally without even
having to configure it first, and use that as the kernel ABI
definition.  You should only have to do this again *if* the ABI
changes (ideally no removals, only additions) *and* you want to take
advantage of them, *and* you're willing to rebuild userland pieces
that could take advantage of them.

On most systems, people won't do that, they'll just use whatever
kernel headers and glibc binaries their distro ships.  If they want to
change any of these, they have to know what they're doing.

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 22:55                             ` Linus Torvalds
@ 2004-11-30 23:29                               ` Matt Mackall
  2004-12-03  1:03                                 ` Krzysztof Halasa
  2004-11-30 23:51                               ` David Woodhouse
  1 sibling, 1 reply; 185+ messages in thread
From: Matt Mackall @ 2004-11-30 23:29 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: David Woodhouse, Alexandre Oliva, Paul Mackerras, Greg KH,
	Matthew Wilcox, David Howells, hch, linux-kernel, libc-hacker

On Tue, Nov 30, 2004 at 02:55:00PM -0800, Linus Torvalds wrote:
> 
> 
> On Tue, 30 Nov 2004, Matt Mackall wrote:
> > 
> > So we follow dhowell's plan with the following additions:
> 
> No.
> 
> We do _not_ move stuff over that is questionable.

Umm. I don't think you even read what I wrote.
 
> I thought that was clear by now. The rules are:
>  - we only move things that _have_ to move

Yes. And when we move a definition from linux/foo.h to user/foo.h, we include
them from linux/foo.h..

>  - we don't break existing programs, and no "but they are broken already" 
>    is not an excuse.

..which prevents userland breakage during the transition (and is generally what
you want in the kernel anyway).

>  - we only move things where that _particular_ move can be shown to be 
>    beneficial.

It's my opinion that it will eventually be deemed beneficial to
separate out everything that has a legitimate reason to be used by
userspace. But I'm not suggesting getting there in one go, what I'm
suggesting is how to get there incrementally. To rehash:

1. create include/user and friends
2. when we run across a troublesome ABI definition:
     create include/user/foo.h and move the definition there
     make sure include/linux/foo.h includes it
     userspace and kernel compile as before
     send a patch
3. repeat step 2 as often as useful
4. add new user ABI always to include/user/
5. if at some point we find that all of userspace builds from
   include/user/ without reference to include/linux/, declare
   include/user the canonical ABI
6. drop #define __KERNEL__, etc..
7. drop any superfluous include/linux -> include/user includes (there
   shouldn't be many)

-- 
Mathematics is the supreme nostalgia of our time.

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 21:39                       ` Alexandre Oliva
@ 2004-11-30 23:33                         ` Matthew Wilcox
  0 siblings, 0 replies; 185+ messages in thread
From: Matthew Wilcox @ 2004-11-30 23:33 UTC (permalink / raw)
  To: Alexandre Oliva
  Cc: Linus Torvalds, David Howells, Paul Mackerras, Greg KH,
	David Woodhouse, Matthew Wilcox, hch, linux-kernel, libc-hacker

On Tue, Nov 30, 2004 at 07:39:34PM -0200, Alexandre Oliva wrote:
> Sure, we could take headers from linux-*/include/user and install them
> in /usr/include/kernel, but then includes in there that reference
> other headers in user/ or in asm-<arch>/ will cease to work.

I covered this with an evil sed script earlier in the thread.

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 23:09                               ` Chris Friesen
@ 2004-11-30 23:35                                 ` Matt Mackall
  2004-12-01  0:23                                 ` Miquel van Smoorenburg
  1 sibling, 0 replies; 185+ messages in thread
From: Matt Mackall @ 2004-11-30 23:35 UTC (permalink / raw)
  To: Chris Friesen
  Cc: Christoph Hellwig, Linus Torvalds, David Woodhouse,
	Alexandre Oliva, Paul Mackerras, Greg KH, Matthew Wilcox,
	David Howells, linux-kernel, libc-hacker

On Tue, Nov 30, 2004 at 05:09:35PM -0600, Chris Friesen wrote:
> Christoph Hellwig wrote:
> 
> >>b) when include/user is deemed sufficiently populated, a flag day is
> >>declared and links from /usr/include are switched to them
> >
> >
> >there are no such links, only copies (more or less modified)

Indeed.
 
> This may be somewhat heretical, but someone has to ask...
> 
> Once include/user/foo.h is sufficiently clean and sufficiently complete, is 
> there any reason to not allow such links?

Briefly it's preferable not to need the kernel source nor a particular
version of the kernel source around to compile things. Though a
complete include/user makes building a proper kernel-headers package
to install in /usr/include pretty trivial.

-- 
Mathematics is the supreme nostalgia of our time.

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 23:03                                   ` Al Viro
  2004-11-30 23:08                                     ` Mariusz Mazur
  2004-11-30 23:13                                     ` Alexandre Oliva
@ 2004-11-30 23:36                                     ` Chris Friesen
  2 siblings, 0 replies; 185+ messages in thread
From: Chris Friesen @ 2004-11-30 23:36 UTC (permalink / raw)
  To: Al Viro
  Cc: Mariusz Mazur, Sam Ravnborg, Linus Torvalds, David Woodhouse,
	Alexandre Oliva, Paul Mackerras, Greg KH, Matthew Wilcox,
	David Howells, hch, linux-kernel, libc-hacker

Al Viro wrote:
> On Tue, Nov 30, 2004 at 11:44:21PM +0100, Mariusz Mazur wrote:

>>Wrong. These dirs must be linked to /usr/include so they must have more 
>>meaningfull names.
> 
> 
> WTF?  I've got a dozen kernel trees hanging around, which one (and WTF any,
> while we are at it) should be "linked to"?

Just my point of view, but I'd love to have /usr/include/linux and 
/usr/include/asm linked to the headers for the currently running kernel.

For a while I was playing with adding a bunch of new syscalls, and it sure would 
have been nice to be able to just include <asm/unistd.h> and get the up-to-date 
syscalls for whatever kernel I happened to be running.

I've since switched to using a device node and doing ioctl() calls to get around 
this, but the issue still stands as I need to either import the ioctl() command 
numbers and data types, or else I have to duplicate the header in userspace and 
keep the two versions up to date.

Chris

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 22:51                           ` Linus Torvalds
@ 2004-11-30 23:50                             ` David Woodhouse
  2004-12-01  0:10                               ` Linus Torvalds
  2004-12-01  2:02                               ` cdrom.h (was Re: [RFC] Splitting kernel headers...) Matthew Wilcox
  0 siblings, 2 replies; 185+ messages in thread
From: David Woodhouse @ 2004-11-30 23:50 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Alexandre Oliva, dhowells, Paul Mackerras, Greg KH,
	Matthew Wilcox, hch, linux-kernel, libc-hacker

On Tue, 2004-11-30 at 14:51 -0800, Linus Torvalds wrote:
> The way to prevent _future_ abuse is by adding something like
> 
> 	#ifndef __KERNEL__
> 	#warning "This really doesn't work"
> 	#endif
> 
> which does that, and has the advantage of not breaking anything at all.
> 
> In other words: if you want to move things around just to break things, 
> THEN THAT IS INCREDIBLY STUPID. We don't do things to screw our users 
> over. 

Linus, you're arguing that it's better to let users use something which
is non-portable and silently does the wrong thing, as long as it
actually compiles. That this is preferable to making sure it doesn't
compile. 

Now, there are cases (like _perhaps_ byteorder.h) where we should
probably allow this kind of 'abuse' to continue because it's fairly
harmless and it does actually _work_.

But atomic.h isn't an example of that.

If I wrote a userspace program which relies upon the md5sum of certain
kernel headers not changing, would you decree that the headers in
question should not change at _all_ because that would 'break existing
software'? Of course not; you have to draw the line somewhere. And I
would draw it somewhere between atomic.h and byteorder.h -- where would
_you_ draw it?

-- 
dwmw2


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 22:55                             ` Linus Torvalds
  2004-11-30 23:29                               ` Matt Mackall
@ 2004-11-30 23:51                               ` David Woodhouse
  1 sibling, 0 replies; 185+ messages in thread
From: David Woodhouse @ 2004-11-30 23:51 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Matt Mackall, Alexandre Oliva, Paul Mackerras, Greg KH,
	Matthew Wilcox, dhowells, hch, linux-kernel, libc-hacker

On Tue, 2004-11-30 at 14:55 -0800, Linus Torvalds wrote:
> 
> On Tue, 30 Nov 2004, Matt Mackall wrote:
> > 
> > So we follow dhowell's plan with the following additions:
> 
> No.
> 
> We do _not_ move stuff over that is questionable.
> 
> I thought that was clear by now. The rules are:
>  - we only move things that _have_ to move
>  - we don't break existing programs, and no "but they are broken already" 
>    is not an excuse.
>  - we only move things where that _particular_ move can be shown to be 
>    beneficial.
> 
> No whole-sale moves. No "let's break things that I think are broken". No 
> "let's change things because we can".
> 
> Well-defined moves. Both in content _and_ in reason.

Fine. And yes, we do it piecemeal with each part being given individual
consideration.

I'd like to aim for a conclusion where everything which userspace needs
is in the new directories and nothing is required from the private
directories any more. But let's start with the really important parts
which really _are_ crying out to be fixed.

-- 
dwmw2


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 23:50                             ` David Woodhouse
@ 2004-12-01  0:10                               ` Linus Torvalds
  2004-12-01  0:18                                 ` Linus Torvalds
  2004-12-01  0:24                                 ` David Woodhouse
  2004-12-01  2:02                               ` cdrom.h (was Re: [RFC] Splitting kernel headers...) Matthew Wilcox
  1 sibling, 2 replies; 185+ messages in thread
From: Linus Torvalds @ 2004-12-01  0:10 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Alexandre Oliva, dhowells, Paul Mackerras, Greg KH,
	Matthew Wilcox, hch, linux-kernel, libc-hacker



On Tue, 30 Nov 2004, David Woodhouse wrote:
> 
> Linus, you're arguing that it's better to let users use something which
> is non-portable and silently does the wrong thing, as long as it
> actually compiles. That this is preferable to making sure it doesn't
> compile. 

I'm saying that if it worked before, it should work after. And my 
suggestion gives a nice warning.

> But atomic.h isn't an example of that.

Even atomic.h. I could well imagine that somebody includes atomic.h just 
to get the thread-safe updates for some architectures. For example, 
asm-alpha/atomic.h does it right, and I woul dnot be at all surprised if 
somebody had noticed.

And your suggestion has the problem that the people who get bitten by a 
non-compiling thing are not necessarily the same people who can fix it.

> software'? Of course not; you have to draw the line somewhere. And I
> would draw it somewhere between atomic.h and byteorder.h -- where would
> _you_ draw it?

I'll draw it at "somebody might validly use it", because the fact is, I 
can't test it. 

Which is why I want patches to this to be OBVIOUSLY CORRECT, dammit! How 
hard is that to understand?

The fact is, the less benefit there is from a change, the more obviously 
correct it has to be in all forms. Moving stuff around in header files 
basically fixes zero bugs in the kernel, so the benefit for the kernel is 
basically zero. Which means that the obviousness factor of the change had 
better be pretty close to infinite.

Comprende? 

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-01  0:10                               ` Linus Torvalds
@ 2004-12-01  0:18                                 ` Linus Torvalds
  2004-12-01  8:10                                   ` Arjan van de Ven
  2004-12-01  0:24                                 ` David Woodhouse
  1 sibling, 1 reply; 185+ messages in thread
From: Linus Torvalds @ 2004-12-01  0:18 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Alexandre Oliva, dhowells, Paul Mackerras, Greg KH,
	Matthew Wilcox, hch, Kernel Mailing List



On Tue, 30 Nov 2004, Linus Torvalds wrote:
> 
> Even atomic.h. I could well imagine that somebody includes atomic.h just 
> to get the thread-safe updates for some architectures. For example, 
> asm-alpha/atomic.h does it right, and I would not be at all surprised if 
> somebody had noticed.

In fact, this is not entirely theoretical. I know people _have_ noticed, 
because I've gotten queries from some projects that wanted to copy the 
definitions for their alpha port. I only got those queries because my name 
is on the file, and those people wanted to actually copy them, not just 
include the header file. I don't know _how_ many people decided to just 
do the #include.

There are probably more people familiar with the kernel source tree than 
with GLIB which is probably the preferred way to do those things these 
days. And a few years ago that choice wasn't even there. 

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 23:09                               ` Chris Friesen
  2004-11-30 23:35                                 ` Matt Mackall
@ 2004-12-01  0:23                                 ` Miquel van Smoorenburg
  1 sibling, 0 replies; 185+ messages in thread
From: Miquel van Smoorenburg @ 2004-12-01  0:23 UTC (permalink / raw)
  To: linux-kernel

In article <41ACFDAF.3040209@nortelnetworks.com>,
Chris Friesen  <cfriesen@nortelnetworks.com> wrote:
>Christoph Hellwig wrote:
>
>>>b) when include/user is deemed sufficiently populated, a flag day is
>>>declared and links from /usr/include are switched to them
>> 
>> 
>> there are no such links, only copies (more or less modified)
>
>This may be somewhat heretical, but someone has to ask...
>
>Once include/user/foo.h is sufficiently clean and sufficiently complete, is 
>there any reason to not allow such links?

It would be cleaner to use gcc -I /lib/modules/`uname -r`/build/include
so that you don't break regular compiles by accident.

Mike.


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-01  0:10                               ` Linus Torvalds
  2004-12-01  0:18                                 ` Linus Torvalds
@ 2004-12-01  0:24                                 ` David Woodhouse
  2004-12-01  0:37                                   ` Linus Torvalds
  1 sibling, 1 reply; 185+ messages in thread
From: David Woodhouse @ 2004-12-01  0:24 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Alexandre Oliva, dhowells, Paul Mackerras, Greg KH,
	Matthew Wilcox, hch, linux-kernel, libc-hacker

On Tue, 2004-11-30 at 16:10 -0800, Linus Torvalds wrote:
> I'll draw it at "somebody might validly use it", because the fact is, I 
> can't test it. 
> 
> Which is why I want patches to this to be OBVIOUSLY CORRECT, dammit! How 
> hard is that to understand?

The concept isn't at all hard to understand. But no patch is 'obviously
correct' if you want to protect against the _slightest_ possibility that
people might be abusing something you're taking away. 

Some people might define __KERNEL__ on purpose when compiling something
in userspace, to get something that would otherwise be hidden from them.
Would you consider that sacrosanct too?

-- 
dwmw2


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-01  0:24                                 ` David Woodhouse
@ 2004-12-01  0:37                                   ` Linus Torvalds
  2004-12-01  0:47                                     ` David Woodhouse
  0 siblings, 1 reply; 185+ messages in thread
From: Linus Torvalds @ 2004-12-01  0:37 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Alexandre Oliva, dhowells, Paul Mackerras, Greg KH,
	Matthew Wilcox, hch, linux-kernel, libc-hacker



On Wed, 1 Dec 2004, David Woodhouse wrote:
> 
> The concept isn't at all hard to understand. But no patch is 'obviously
> correct' if you want to protect against the _slightest_ possibility that
> people might be abusing something you're taking away. 

I really disagree. That's kind of my point. We _can_ make sure that there 
is abzolutely zero semantic content change.

People both inside and outside the kernel who use the old <linux/xxx.h>
headers will get exactly what they got before if we do it right.

> Some people might define __KERNEL__ on purpose when compiling something
> in userspace, to get something that would otherwise be hidden from them.
> Would you consider that sacrosanct too?

Why _do_ you want to break things? Do the cleanup. Don't do the breakage.

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-01  0:37                                   ` Linus Torvalds
@ 2004-12-01  0:47                                     ` David Woodhouse
  2004-12-01  0:57                                       ` Linus Torvalds
  0 siblings, 1 reply; 185+ messages in thread
From: David Woodhouse @ 2004-12-01  0:47 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Alexandre Oliva, dhowells, Paul Mackerras, Greg KH,
	Matthew Wilcox, hch, linux-kernel, libc-hacker

On Tue, 2004-11-30 at 16:37 -0800, Linus Torvalds wrote:
> 
> On Wed, 1 Dec 2004, David Woodhouse wrote:
> > 
> > The concept isn't at all hard to understand. But no patch is 'obviously
> > correct' if you want to protect against the _slightest_ possibility that
> > people might be abusing something you're taking away. 
> 
> I really disagree. That's kind of my point. We _can_ make sure that there 
> is abzolutely zero semantic content change.

We've _never_ made sure that there's absolutely zero semantic content
change in our private headers.

> People both inside and outside the kernel who use the old <linux/xxx.h>
> headers will get exactly what they got before if we do it right.

They'll get that if we change nothing at all.

> > Some people might define __KERNEL__ on purpose when compiling something
> > in userspace, to get something that would otherwise be hidden from them.
> > Would you consider that sacrosanct too?
> 
> Why _do_ you want to break things? Do the cleanup. Don't do the breakage.

I'm trying to understand precisely where _you_ want to draw the line
between 'cleanup' and 'breakage'. 

You obviously don't agree that including atomic.h was an entirely stupid
thing for userspace to be doing in the first place, and that it's
reasonable for that not to compile in future; you believe that it should
continue to compile and silently do the wrong thing. We've covered that
much.

So where _do_ you draw the line? Trying to use spinlock.h? Defining
__KERNEL__ before including kernel headers?

I can try to apply common sense here, having discussed it with a bunch
of other people. But you don't seem to agree with that; hence further
guidance would be appreciated. But we can just submit patches to do the
cleanups and see what you take and what you don't, if you really want to
make us jump through hoops for the sake of it without indicating what
you'd accept and what you'd not beforehand.

-- 
dwmw2


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-01  0:47                                     ` David Woodhouse
@ 2004-12-01  0:57                                       ` Linus Torvalds
  2004-12-01  1:06                                         ` David Woodhouse
  0 siblings, 1 reply; 185+ messages in thread
From: Linus Torvalds @ 2004-12-01  0:57 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Alexandre Oliva, dhowells, Paul Mackerras, Greg KH,
	Matthew Wilcox, hch, linux-kernel, libc-hacker



On Wed, 1 Dec 2004, David Woodhouse wrote:
>
> > I really disagree. That's kind of my point. We _can_ make sure that there 
> > is abzolutely zero semantic content change.
> 
> We've _never_ made sure that there's absolutely zero semantic content
> change in our private headers.

Not true. For example, I don't take spelling fixes that do semantic 
content changes.

See? The more trivial the fix, the more obviously correct it has to be.

This isn't even a "fix". It's a cleanup. It goes under the same rules a 
spelling fix does.

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-01  0:57                                       ` Linus Torvalds
@ 2004-12-01  1:06                                         ` David Woodhouse
  2004-12-01  1:23                                           ` Linus Torvalds
  2004-12-01  6:48                                           ` BAIN
  0 siblings, 2 replies; 185+ messages in thread
From: David Woodhouse @ 2004-12-01  1:06 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Alexandre Oliva, dhowells, Paul Mackerras, Greg KH,
	Matthew Wilcox, hch, linux-kernel, libc-hacker

On Tue, 2004-11-30 at 16:57 -0800, Linus Torvalds wrote:
> This isn't even a "fix". It's a cleanup. It goes under the same rules
> a spelling fix does.

So you don't see a long-term technical benefit in cleaning up the
API/ABI we export to userspace so that userspace stops depending on
stuff which just isn't supposed to be there? It's all just cosmetic
masturbation as far as you're concerned? There's no point in trying to
get to the point where we don't need to separately maintain a 
glibc-kernheaders package because it can be taken directly from the
kernel?

-- 
dwmw2


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-01  1:06                                         ` David Woodhouse
@ 2004-12-01  1:23                                           ` Linus Torvalds
  2004-12-01  6:48                                           ` BAIN
  1 sibling, 0 replies; 185+ messages in thread
From: Linus Torvalds @ 2004-12-01  1:23 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Alexandre Oliva, dhowells, Paul Mackerras, Greg KH,
	Matthew Wilcox, hch, linux-kernel, libc-hacker



On Wed, 1 Dec 2004, David Woodhouse wrote:
>
> On Tue, 2004-11-30 at 16:57 -0800, Linus Torvalds wrote:
> > This isn't even a "fix". It's a cleanup. It goes under the same rules
> > a spelling fix does.
> 
> So you don't see a long-term technical benefit in cleaning up the
> API/ABI we export to userspace so that userspace stops depending on
> stuff which just isn't supposed to be there?

I see _benefits_, but exactly because they aren't immediate, I think we 
should be careful.

Also, exactly because moving things around is a total _nightmare_ from a 
diff and source maintenance perspective, we should be _extra_ careful. 
It's almost impossible to see something that changed when that something 
is _also_ being moved around. You don't see it in the diff, and you don't 
see it in any of the tools - a small semantic change that would have been 
absolutely OBVIOUS if it was in a regular diff gets totally drowned out by 
the "move" diff. 

Put another way: when people do things like re-indenting a driver (which 
has _exactly_ the same issues: cleanup for the future, but totally 
destroys all chance of seeing what the changes actually were), I want the 
patch that does the re-indentation to be totally independent of the actual 
changes.

Ie we do one patch that _only_ does the whitespace changes, and try to 
make sure that it doesn't break anything even by mistake. People even 
compile the thing and verify that it is 100% the same thing as an object 
file. Then that gets applied. Only _after_ that should you make changes.

Do we follow this religiously? For small things, clearly no. We sometimes 
mix whitespace and real changes. But we sure as hell shouldn't. At least 
not for anything bigger.

The EXACT same thing is true of any header file movement. DO NOT CHANGE 
SEMANTICS WHEN YOU MOVE STUFF!

It's not just my personal hangup. It's a damn good idea.

I will hereby just ignore this thread. Please remove me from the Cc, I've 
wasted way too much time on this total idiocy already. I've made my 
opinions extremely clear, and quite frankly, I don't want to hear anything 
on this any more for at least a week. It's just not worth it.

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* cdrom.h (was Re: [RFC] Splitting kernel headers...)
  2004-11-30 23:50                             ` David Woodhouse
  2004-12-01  0:10                               ` Linus Torvalds
@ 2004-12-01  2:02                               ` Matthew Wilcox
  1 sibling, 0 replies; 185+ messages in thread
From: Matthew Wilcox @ 2004-12-01  2:02 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Linus Torvalds, Alexandre Oliva, dhowells, Paul Mackerras,
	Greg KH, Matthew Wilcox, hch, linux-kernel, libc-hacker

On Tue, Nov 30, 2004 at 11:50:56PM +0000, David Woodhouse wrote:
> Now, there are cases (like _perhaps_ byteorder.h) where we should
> probably allow this kind of 'abuse' to continue because it's fairly
> harmless and it does actually _work_.

OK, time for my pet peeve: linux/cdrom.h.

#include <asm/byteorder.h>

[...]

/* for CDROM_PACKET_COMMAND ioctl */
struct cdrom_generic_command
{
[...]
        struct request_sense    __user *sense;
[...]
}

struct request_sense {
#if defined(__BIG_ENDIAN_BITFIELD)
        __u8 valid              : 1;
        __u8 error_code         : 7;
#elif defined(__LITTLE_ENDIAN_BITFIELD)
        __u8 error_code         : 7;
        __u8 valid              : 1;
#endif
        __u8 segment_number;
[...]
}

At least one architecture's asm/byteorder.h has been C++-incompatible
in the past (I forget what; it was 2 years ago that I cared about it).
So to fix this, we need a asm-*/user/byteorder.h that only describes
the endianness.  Except ... what about mips/mipsel?

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 23:08                                     ` Mariusz Mazur
@ 2004-12-01  5:23                                       ` Sam Ravnborg
  2004-12-01 10:52                                         ` Mariusz Mazur
  0 siblings, 1 reply; 185+ messages in thread
From: Sam Ravnborg @ 2004-12-01  5:23 UTC (permalink / raw)
  To: Mariusz Mazur
  Cc: Al Viro, Sam Ravnborg, Linus Torvalds, David Woodhouse,
	Alexandre Oliva, Paul Mackerras, Greg KH, Matthew Wilcox,
	David Howells, hch, linux-kernel, libc-hacker

On Wed, Dec 01, 2004 at 12:08:13AM +0100, Mariusz Mazur wrote:
> On ?roda 01 grudzie? 2004 00:03, Al Viro wrote:
> > > Wrong. These dirs must be linked to /usr/include so they must have more
> > > meaningfull names.
> >
> > WTF?  I've got a dozen kernel trees hanging around, which one (and WTF any,
> > while we are at it) should be "linked to"?
> 
> Linked, copied, mount --binded, whatever. Just not under the 
> name /usr/include/user, but something more meaningfull.

Whats wrong with
/lib/modules/`uname -r`/source/include/user
/lib/modules/`uname -r`/source/include/$arch

	Sam

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 20:47                             ` Linus Torvalds
  2004-11-30 22:33                               ` Sam Ravnborg
@ 2004-12-01  5:50                               ` H. Peter Anvin
  1 sibling, 0 replies; 185+ messages in thread
From: H. Peter Anvin @ 2004-12-01  5:50 UTC (permalink / raw)
  To: linux-kernel

Followup to:  <Pine.LNX.4.58.0411301243000.22796@ppc970.osdl.org>
By author:    Linus Torvalds <torvalds@osdl.org>
In newsgroup: linux.dev.kernel
> 
> But claiming that __KERNEL__ doesn't work is clearly a bunch of crapola. 
> As is the notion that you can somehow do it all. We do it in small pieces.
> 

IMNSHO, based on my experience with klibc, the problem is not #ifdef
__KERNEL__, the *specific* problems are:

- The kernel exporting libc4/5 internals, which are neither what the
  kernel nor any kind of modern userspace wants.  This should be
  possible to get rid of easily enough.

- The kernel exporting things into the userspace namespace.  The
  kernel's definition of "struct stat" isn't usable -- the one that is
  useful is called "struct stat64" on most, but not all,
  architectures, but because it's a structure tag it can't be pulled
  from the exported kernel ABI in such a way that what userspace calls
  "struct stat" is what the kernel currently calls "struct stat64" if
  that exists.

  This one is a bit uglier, since it probably needs something like:

  #ifdef __KERNEL__
  #define __kabi_stat64 stat64
  #endif

  struct __kabi_stat64 { ... };

	-hpa

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-01  1:06                                         ` David Woodhouse
  2004-12-01  1:23                                           ` Linus Torvalds
@ 2004-12-01  6:48                                           ` BAIN
  1 sibling, 0 replies; 185+ messages in thread
From: BAIN @ 2004-12-01  6:48 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Linus Torvalds, Alexandre Oliva, dhowells, Paul Mackerras,
	Greg KH, Matthew Wilcox, hch, linux-kernel, libc-hacker

On Wed, 01 Dec 2004 01:06:17 +0000, David Woodhouse <dwmw2@infradead.org> wrote:
> On Tue, 2004-11-30 at 16:57 -0800, Linus Torvalds wrote:
> > This isn't even a "fix". It's a cleanup. It goes under the same rules
> > a spelling fix does.
> 
> So you don't see a long-term technical benefit in cleaning up the
> API/ABI we export to userspace so that userspace stops depending on
> stuff which just isn't supposed to be there? It's all just cosmetic
> masturbation as far as you're concerned? There's no point in trying to
> get to the point where we don't need to separately maintain a
> glibc-kernheaders package because it can be taken directly from the
> kernel?

There obviously _is_ a technical benefit. And to put out second usual
argument used to reject patches, this benefit is desirable/matters to
more than single digit number of users.

But lets face it, the changes you guies are discussing in this long
thread are _massive_. May be not so much theorotical mass. In theory
this is just a trivial movement of text from file to file with new
rules enforced on adding new text.
But this does boil down to a great number of user interfaces being
touched. And as far as the linux engineering practices go, this is
_huge_.

The costs are high, each header file change is going to affect
everything, the regression testing is going to be pain if we actually
go do the modifications suggested by this thread.

And lets face it, with a retard[1] chosen as our leader , _mistakes
will happen_. Things will _break_ and any decision taken will more
often be a false one.

But we already know a better way to do things. After all thats how we
have taken linux to the place where it is today. We can just start
making the changes. In small steps. See if anyone is really bothered
by the change. See if enough people give you feedback so that you are
bothered to propagate the change to newer level.

I would much rather like to see the glibc-kernelheaders shrunk by few
bytes every 3-4 months or so. And yes this actually has a chance that
it may never happen.

The whole "put forward an idea get a consensus" and then "try to
engineer it" works well for theory. But not in practise. In the end,
only the good things prevail not because they were created good, but
more because bad things can't servive. Leave the whole idea of "putting
a research together and try to do the ultimate right" for the FreeBSD
team.

And don't try to do the ultimate right without any reasearch, thats
microsoft's domain.

In summary, splitting the headers could be a good idea. But to have
any acceptance you need to prove it by practical examples. And when
you do new things, be prepared to accept that they might fail in
practise.

BAIN

[1]see: 
http://marc.free.net.ph/message/20041109.161141.2f4e1246.html

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-01  0:18                                 ` Linus Torvalds
@ 2004-12-01  8:10                                   ` Arjan van de Ven
  0 siblings, 0 replies; 185+ messages in thread
From: Arjan van de Ven @ 2004-12-01  8:10 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: David Woodhouse, Alexandre Oliva, dhowells, Paul Mackerras,
	Greg KH, Matthew Wilcox, hch, Kernel Mailing List

On Tue, 2004-11-30 at 16:18 -0800, Linus Torvalds wrote:
> On Tue, 30 Nov 2004, Linus Torvalds wrote:
> > 
> > Even atomic.h. I could well imagine that somebody includes atomic.h just 
> > to get the thread-safe updates for some architectures. For example, 
> > asm-alpha/atomic.h does it right, and I would not be at all surprised if 
> > somebody had noticed.
> 
> In fact, this is not entirely theoretical.

indeed it's not. THere were projects that used the x86 atomic.h
header.... but failed to notice that its not actually using the lock
prefix if you don't define CONFIG_SMP. Now it's purely up to luck if the
userspace headers in /usr/include have CONFIG_SMP defined or not, so
there have been cases where people compiled on an UP box and got racy
code. Was a production level project; normally they built on SMP but
once they built on UP .. poof ;)

so atomic.h is *dangerous* for userspace to use, at least on x86. It
gives a false feeling of atomicity.

So dangerous that for the RH/Fedora distros, we don't ship the
questionable parts of it.



^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-01  5:23                                       ` Sam Ravnborg
@ 2004-12-01 10:52                                         ` Mariusz Mazur
  2004-12-01 18:08                                           ` Sam Ravnborg
  0 siblings, 1 reply; 185+ messages in thread
From: Mariusz Mazur @ 2004-12-01 10:52 UTC (permalink / raw)
  To: Sam Ravnborg
  Cc: Al Viro, Linus Torvalds, David Woodhouse, Alexandre Oliva,
	Paul Mackerras, Greg KH, Matthew Wilcox, David Howells, hch,
	linux-kernel, libc-hacker

On środa 01 grudzień 2004 06:23, Sam Ravnborg wrote:
> > Linked, copied, mount --binded, whatever. Just not under the
> > name /usr/include/user, but something more meaningfull.
>
> Whats wrong with
> /lib/modules/`uname -r`/source/include/user
> /lib/modules/`uname -r`/source/include/$arch

Those are supposed to be userland-only headers that don't just change - they 
are gradually expanded. I don't see a point in having `uname -r` in there.

And another thing - distribution vendors will *hate* anyone, that encourages 
app developers to add an include path based on which kernel is being 
currently run. People, that have their headers tied to their kernels are a 
*minority*.

(though iirc eg. netfilter is a mess, so there might be bigger reasons to 
ignore my arguments; not that I won't hate you if you do, and I'll have to 
patch everything, just to get it compiled)

-- 
In the year eighty five ten
God is gonna shake his mighty head
He'll either say,
"I'm pleased where man has been"
Or tear it down, and start again

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29 22:44               ` Al Viro
@ 2004-12-01 11:36                 ` Roman Zippel
  0 siblings, 0 replies; 185+ messages in thread
From: Roman Zippel @ 2004-12-01 11:36 UTC (permalink / raw)
  To: Al Viro
  Cc: Linus Torvalds, Alexandre Oliva, Paul Mackerras, Greg KH,
	David Woodhouse, Matthew Wilcox, David Howells, hch,
	linux-kernel, libc-hacker

Hi,

On Mon, 29 Nov 2004, Al Viro wrote:

> That leaves affs_fs.h containing only AFFS_SUPER_MAGIC declaration and
> amigaffs.h also seriously cleaned up; it still has a bunch of defines that
> are almost certainly never used in userland (they refer to ->b_data and
> in-core affs superblock), but these might in theory be useful (similar
> stuff for ext2 *is* used by userland code).
> 
> If Roman is OK with their removal, AFFS_DATA and friends should also get
> moved to fs/affs/affs.h.

Looks good. Thanks.
I wouldn't mind moving it all to fs/affs, I don't think there is any 
userspace dependency here. I have a simple mkaffs tool, but that uses a 
copy of it.

> +#if 0
> +	s32	 i_original;			/* if != 0, this is the key of the original */
> +	u32	 i_data[AFFS_MAX_PREALLOC];	/* preallocated blocks */
> +	int	 i_cache_users;			/* Cache cannot be freed while > 0 */
> +	unsigned char i_hlink;			/* This is a fake */
> +	unsigned char i_pad;
> +	s32	 i_parent;			/* parent ino */
> +#endif

That can be killed.

bye, Roman

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29  9:42           ` David Woodhouse
  2004-11-30  8:08             ` Alex Riesen
@ 2004-12-01 11:46             ` Ralf Baechle
  2004-12-02  1:21               ` H. Peter Anvin
  1 sibling, 1 reply; 185+ messages in thread
From: Ralf Baechle @ 2004-12-01 11:46 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Linus Torvalds, Paul Mackerras, Greg KH, Matthew Wilcox,
	David Howells, hch, aoliva, linux-kernel, libc-hacker

On Mon, Nov 29, 2004 at 09:42:16AM +0000, David Woodhouse wrote:

> I've lost track of the number of times things have broken because of
> incorrect use of kernel headers from userspace. That's what we're trying
> to fix -- by putting only the bits which are _supposed_ to be visible
> into files which userspace sees, where we know they define part of the
> userspace API and hence we can be extremely careful when editing them. 
> 
> I don't think it makes sense at this point for us to bury our collective
> heads in the sand and pretend there isn't a problem here that's worth
> fixing.
> 
> I agree that it should be obviously correct though -- and that's why
> we're trying to end up with a structure that in the first pass would
> give us in userspace essentially what we already have in the various
> glibc-kernheaders packages, but without the constant and unnecessary
> need for some poor sod to keep those up to date by hand.

The concept of copying kernel headers into applications is even worse
when arch portability is affected.  I stopped counting how often I had
to fix that kind of crap - and the state of kernel headers and userspace
kernel header packages is really provoking that kind of mess.

  Ralf

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-01 10:52                                         ` Mariusz Mazur
@ 2004-12-01 18:08                                           ` Sam Ravnborg
  0 siblings, 0 replies; 185+ messages in thread
From: Sam Ravnborg @ 2004-12-01 18:08 UTC (permalink / raw)
  To: Mariusz Mazur
  Cc: Sam Ravnborg, Al Viro, Linus Torvalds, David Woodhouse,
	Alexandre Oliva, Paul Mackerras, Greg KH, Matthew Wilcox,
	David Howells, hch, linux-kernel, libc-hacker

On Wed, Dec 01, 2004 at 11:52:45AM +0100, Mariusz Mazur wrote:
> On ?roda 01 grudzie? 2004 06:23, Sam Ravnborg wrote:
> > > Linked, copied, mount --binded, whatever. Just not under the
> > > name /usr/include/user, but something more meaningfull.
> >
> > Whats wrong with
> > /lib/modules/`uname -r`/source/include/user
> > /lib/modules/`uname -r`/source/include/$arch
> 
> Those are supposed to be userland-only headers that don't just change - they 
> are gradually expanded. I don't see a point in having `uname -r` in there.
> 
> And another thing - distribution vendors will *hate* anyone, that encourages 
> app developers to add an include path based on which kernel is being 
> currently run. People, that have their headers tied to their kernels are a 
> *minority*.
Above shows how this minority could locate headers for running kernel.
It is not meant to be the general solution.

	Sam

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 23:12                         ` Linus Torvalds
@ 2004-12-01 19:41                           ` Alexandre Oliva
  2004-12-01 19:54                             ` Matthew Wilcox
  2004-12-02  3:58                             ` Linus Torvalds
  0 siblings, 2 replies; 185+ messages in thread
From: Alexandre Oliva @ 2004-12-01 19:41 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox,
	David Howells, hch, linux-kernel, libc-hacker

On Nov 30, 2004, Linus Torvalds <torvalds@osdl.org> wrote:

> On Tue, 30 Nov 2004, Alexandre Oliva wrote:

>> On Nov 30, 2004, Linus Torvalds <torvalds@osdl.org> wrote:

>> > If you want to use a legal analogy, the ABI is not a contract, it's a
>> > public _license_.

>> I didn't mean to use a legal analogy.  I meant contract in the
>> software engineering sense.  Sorry if that wasn't clear.

> Then your definition of a "contract" is flawed or your world-view has 
> nothing to do with reality.

It's not my definition, it's a definition used in software
engineering.  Sure, if you ask a legal mind, you'll get a different
answer.  If you ask a bridge player, you'll get yet another
definition.  That's why we have jargons in which common-use words have
their meanings narrowed to the specifics of certain fields.  And
that's why it's so hard for specialists in different fields to talk to
each other at times when there's disagreement about the precise
meaning of certain words.

> An ABI is not a contract.

Not a legal contract, for sure.

An ABI is a definition of an interface, including operations with pre-
and post-conditions, data structures with their invariants, constants,
file formats, etc.  Most of that is covered by the software
engineering term `contract'.

Sure enough, the headers we're talking about describe only a very
small portion on that; pretty much only data structures and
constants.  It certainly doesn't *specify* the entire ABI, it only
highlights portions of it.



Anyhow, all of this is beyond the point.  I see you've decreed that
people can introduce `user' directories in the kernel now.  Would you
please reconsider and choose a dir name that would enable the same ABI
headers to be used by kernel and userland, without adding a directory
to /usr/include that has no indication that it comes from the kernel?

Thanks,

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-01 19:41                           ` Alexandre Oliva
@ 2004-12-01 19:54                             ` Matthew Wilcox
  2004-12-02  3:58                             ` Linus Torvalds
  1 sibling, 0 replies; 185+ messages in thread
From: Matthew Wilcox @ 2004-12-01 19:54 UTC (permalink / raw)
  To: Alexandre Oliva
  Cc: Linus Torvalds, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, David Howells, hch, linux-kernel, libc-hacker

On Wed, Dec 01, 2004 at 05:41:25PM -0200, Alexandre Oliva wrote:
> Anyhow, all of this is beyond the point.  I see you've decreed that
> people can introduce `user' directories in the kernel now.  Would you
> please reconsider and choose a dir name that would enable the same ABI
> headers to be used by kernel and userland, without adding a directory
> to /usr/include that has no indication that it comes from the kernel?

As I've said before, this can be fixed with a sed script.

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-01 11:46             ` Ralf Baechle
@ 2004-12-02  1:21               ` H. Peter Anvin
  0 siblings, 0 replies; 185+ messages in thread
From: H. Peter Anvin @ 2004-12-02  1:21 UTC (permalink / raw)
  To: linux-kernel

Followup to:  <20041201114622.GB4876@linux-mips.org>
By author:    Ralf Baechle <ralf@linux-mips.org>
In newsgroup: linux.dev.kernel
> 
> The concept of copying kernel headers into applications is even worse
> when arch portability is affected.  I stopped counting how often I had
> to fix that kind of crap - and the state of kernel headers and userspace
> kernel header packages is really provoking that kind of mess.
> 

Indeed.  The major reason why we need the kernel to export its own ABI
information is because there are at my last counting 18 architectures
running stock Linux, and each one of them has *at least* one ABI, and
they're all different.

	-hpa

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-01 19:41                           ` Alexandre Oliva
  2004-12-01 19:54                             ` Matthew Wilcox
@ 2004-12-02  3:58                             ` Linus Torvalds
  2004-12-02  4:28                               ` Bernd Eckenfels
  2004-12-02  9:29                               ` Pekka Enberg
  1 sibling, 2 replies; 185+ messages in thread
From: Linus Torvalds @ 2004-12-02  3:58 UTC (permalink / raw)
  To: Alexandre Oliva
  Cc: Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox,
	David Howells, hch, linux-kernel, libc-hacker



On Wed, 1 Dec 2004, Alexandre Oliva wrote:
> 
> > Then your definition of a "contract" is flawed or your world-view has 
> > nothing to do with reality.
> 
> It's not my definition, it's a definition used in software
> engineering. 

Not that I've seen. Maybe your school. Google also doesn't seem to agree 
with you. In other words, it seems to be in pretty limited use.

> An ABI is a definition of an interface, including operations with pre-
> and post-conditions, data structures with their invariants, constants,
> file formats, etc.

Oh, I know what an ABI is. I just don't think your "contract" part has 
anything to do with it. 

> Most of that is covered by the software engineering term `contract'.

I think you're making that up. Maybe there's some sw cult that swears by 
"contract programming", the same way there are the "extreme programming" 
cults etc.  For example, I find this "Design by Contract" cult for object- 
oriented programming, but it has _zero_ to do with external API's, and is 
all about the interfaces for object-oriented components.

IOW, I don't find your arguments or your language usage in the least 
convincing. But hey, I did all my CS stuff outside of the US, whatever.

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-02  3:58                             ` Linus Torvalds
@ 2004-12-02  4:28                               ` Bernd Eckenfels
  2004-12-02  5:11                                 ` Peter Williams
  2004-12-02  9:29                               ` Pekka Enberg
  1 sibling, 1 reply; 185+ messages in thread
From: Bernd Eckenfels @ 2004-12-02  4:28 UTC (permalink / raw)
  To: linux-kernel

In article <Pine.LNX.4.58.0412011948450.22796@ppc970.osdl.org> you wrote:
> I think you're making that up. Maybe there's some sw cult that swears by 
> "contract programming"

Design by Contract is the reason for descibing the agreement between
caller/callee as an contract. Bertran Meyer added the pre-conditions and
post-conditions (kind of asserts) to his Eiffel Language, and I dont think
that that is limited to a single system, but is also valid for bondaries
like an ABI. It describes conventions like syscall numbers, too.

Greetings
Bernd

PS: http://en.wikipedia.org/wiki/Eiffel_programming_language

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-02  4:28                               ` Bernd Eckenfels
@ 2004-12-02  5:11                                 ` Peter Williams
  2004-12-02 11:30                                   ` Tomas Carnecky
  0 siblings, 1 reply; 185+ messages in thread
From: Peter Williams @ 2004-12-02  5:11 UTC (permalink / raw)
  To: Bernd Eckenfels; +Cc: linux-kernel

Bernd Eckenfels wrote:
> In article <Pine.LNX.4.58.0412011948450.22796@ppc970.osdl.org> you wrote:
> 
>>I think you're making that up. Maybe there's some sw cult that swears by 
>>"contract programming"
> 
> 
> Design by Contract is the reason for descibing the agreement between
> caller/callee as an contract.

Design by contract isn't really an agreement between the caller and the 
callee (which may not even exist when the contract is created).  It's 
more of a (one way) promise by the callee that if the interface is used 
as described in the contract that it will correctly perform the 
advertised operation (where the advertised operation generally includes 
descriptions of possible failures and how they will be signalled).  Most 
C APIs meet these criteria even though their pre and post conditions are 
expressed less formally than an Eiffel interface.

> Bertran Meyer added the pre-conditions and
> post-conditions (kind of asserts) to his Eiffel Language, and I dont think
> that that is limited to a single system, but is also valid for bondaries
> like an ABI. It describes conventions like syscall numbers, too.

I agree but think it's important to realize that it's a unilateral 
promise on the part of the callee (rather than agreement between the 
callee and the caller) which is in accord with Linus's view of the ABI.

Peter
-- 
Peter Williams                                   pwil3058@bigpond.net.au

"Learning, n. The kind of ignorance distinguishing the studious."
  -- Ambrose Bierce

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-02  3:58                             ` Linus Torvalds
  2004-12-02  4:28                               ` Bernd Eckenfels
@ 2004-12-02  9:29                               ` Pekka Enberg
  2004-12-02 15:35                                 ` Linus Torvalds
  1 sibling, 1 reply; 185+ messages in thread
From: Pekka Enberg @ 2004-12-02  9:29 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Alexandre Oliva, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, David Howells, hch, linux-kernel, libc-hacker,
	penberg

Hi Linus,

[I am getting off-topic but...]

On Wed, 1 Dec 2004, Alexandre Oliva wrote:
> > Most of that is covered by the software engineering term `contract'.

On Wed, Linus Torvalds wrote:
> I think you're making that up. Maybe there's some sw cult that swears by
> "contract programming", the same way there are the "extreme programming"
> cults etc.  For example, I find this "Design by Contract" cult for object-
> oriented programming, but it has _zero_ to do with external API's, and is
> all about the interfaces for object-oriented components.

How is a class interface different from an external API? Sure, you
don't want to change a _published_ API too often but I fail to see how
internal and external APIs are fundamentally different.

I think Alexandre's definition has in fact originated from the "Design
by Contract" cult. And it's all pretty simple stuff:

  - You need to call a function in a certain way (precondition). The
caller fills this part
    of the contract. Now the caller can _expect_ something from the
function (see below).
  - The function does what it has promised to do and optionally
returns some values
    (postcondition). The callee fills this part of the contract.
  - The function expects some parts of the system state to remain stable during
    execution (invariants). In the kernel, you use BUG_ON for these btw.

So there's nothing new here really. However, if your _tools_ support
Design by Contract, you can be explicit about this and enforce the
'contract' during compile-time or run-time.

And I think you're already doing this with your "require spinlock to
be taken" sparse thingy...

On Wed, Linus Torvalds wrote:
> IOW, I don't find your arguments or your language usage in the least
> convincing. But hey, I did all my CS stuff outside of the US, whatever.

I don't think it's an US thing. At the university you did your CS
stuff, we (well at least I) use the term 'contract' pretty much the
same way Alexandre does...

                              Pekka

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-02  5:11                                 ` Peter Williams
@ 2004-12-02 11:30                                   ` Tomas Carnecky
  2004-12-02 22:17                                     ` Peter Williams
  0 siblings, 1 reply; 185+ messages in thread
From: Tomas Carnecky @ 2004-12-02 11:30 UTC (permalink / raw)
  To: Peter Williams; +Cc: Bernd Eckenfels, linux-kernel

Peter Williams wrote:
> Design by contract isn't really an agreement between the caller and the 
> callee (which may not even exist when the contract is created).  It's 
> more of a (one way) promise by the callee that if the interface is used 
> as described in the contract that it will correctly perform the 
> advertised operation (where the advertised operation generally includes 
> descriptions of possible failures and how they will be signalled).  Most 
> C APIs meet these criteria even though their pre and post conditions are 
> expressed less formally than an Eiffel interface.

Design by Contract, as seen in the Eiffel language, is not a one way 
promise, it's a contract between the caller and callee. Both sides have
to fullfil their part of the contract, the caller has to make sure that
the input are valid, and (only) given that, the callee can/has to make
sure that the caller gets the right output.

BTW, Bertrand Meyer is one of my professors, I should know the Eiffel 
language... :)

> I agree but think it's important to realize that it's a unilateral 
> promise on the part of the callee (rather than agreement between the 
> callee and the caller) which is in accord with Linus's view of the ABI.

Whenever you have two sides/parties, you have to agree on _something_, 
otherwise you can't communicate, if you speak to someone, you have to 
choose a language in which to speak, that's the first agreement.
In this situation you have two sides, the kernel and the userspace. Your 
first agreement is the syscall number, and then the arguments, the type 
and format of the arguments, etc. Both sides have to agree on those things.

tom



^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-02  9:29                               ` Pekka Enberg
@ 2004-12-02 15:35                                 ` Linus Torvalds
  2004-12-02 21:46                                   ` Pekka Enberg
  0 siblings, 1 reply; 185+ messages in thread
From: Linus Torvalds @ 2004-12-02 15:35 UTC (permalink / raw)
  To: Pekka Enberg
  Cc: Alexandre Oliva, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, David Howells, hch, linux-kernel, libc-hacker,
	penberg



On Thu, 2 Dec 2004, Pekka Enberg wrote:
> 
> How is a class interface different from an external API?

It's different from an external API exactly because it's not "external". 

I dunno about you, but the (admittedly few) explicitly object-oriented 
projects I've seen from the inside, the class interface is something that 
you consider internal to your project, and are willing to change.

In Linux, the VFS layer comes pretty close: it's really very much about
object-oriented programming, except the implementation is explicit rather
than left up to the programmer. And we have very explicit rules on how
you're supposed to do things: some are even _documented_ (oh, horrors) in
Documentation/filesystems/vfs.txt.

And yes, in this kind of environment I actually think "contract" makes 
some kind of sense, because it is an internal thing that _has_ come out of 
agreement on both sides, and where changes need to be agreed upon by both 
user and definition. It's not an a-priori interface, it's something where 
both sides are intimately involved.

> I think Alexandre's definition has in fact originated from the "Design
> by Contract" cult. And it's all pretty simple stuff:
> 
>   - You need to call a function in a certain way (precondition). The
>     caller fills this part
>     of the contract. Now the caller can _expect_ something from the
>     function (see below).
>   - The function does what it has promised to do and optionally
>     returns some values
>     (postcondition). The callee fills this part of the contract.
>   - The function expects some parts of the system state to remain stable during
>     execution (invariants). In the kernel, you use BUG_ON for these btw.
> 
> So there's nothing new here really. However, if your _tools_ support
> Design by Contract, you can be explicit about this and enforce the
> 'contract' during compile-time or run-time.
> 
> And I think you're already doing this with your "require spinlock to
> be taken" sparse thingy...

Yes. And I think it makes sense _internally_ for a project. 

But pretty much by definition, that's not an ABI. That's an "internal 
interfaces" kind of thing.

> On Wed, Linus Torvalds wrote:
> > IOW, I don't find your arguments or your language usage in the least
> > convincing. But hey, I did all my CS stuff outside of the US, whatever.
> 
> I don't think it's an US thing. At the university you did your CS
> stuff, we (well at least I) use the term 'contract' pretty much the
> same way Alexandre does...

It may also be a new thing. Dammit, when I started, we had USCD Pascal,
and we were happy. None of this OO stuff ;)

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-02 15:35                                 ` Linus Torvalds
@ 2004-12-02 21:46                                   ` Pekka Enberg
  0 siblings, 0 replies; 185+ messages in thread
From: Pekka Enberg @ 2004-12-02 21:46 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Pekka Enberg, Alexandre Oliva, Paul Mackerras, Greg KH,
	David Woodhouse, Matthew Wilcox, David Howells, hch,
	linux-kernel, libc-hacker

On Thu, 2 Dec 2004, Pekka Enberg wrote:
> > How is a class interface different from an external API?

On Thu, 2004-12-02 at 07:35 -0800, Linus Torvalds wrote:
> It's different from an external API exactly because it's not "external". 
> 
> I dunno about you, but the (admittedly few) explicitly object-oriented 
> projects I've seen from the inside, the class interface is something that 
> you consider internal to your project, and are willing to change.
> 
> In Linux, the VFS layer comes pretty close: it's really very much about
> object-oriented programming, except the implementation is explicit rather
> than left up to the programmer. And we have very explicit rules on how
> you're supposed to do things: some are even _documented_ (oh, horrors) in
> Documentation/filesystems/vfs.txt.
> 
> And yes, in this kind of environment I actually think "contract" makes 
> some kind of sense, because it is an internal thing that _has_ come out of 
> agreement on both sides, and where changes need to be agreed upon by both 
> user and definition. It's not an a-priori interface, it's something where 
> both sides are intimately involved.

I tend to think the other way around actually. For published (external)
APIs, contracts are far more important because you cannot review or test
the callers. For internal APIs, contracts can be more relaxed as you
usually have full control over the codebase. So no, I don't think
contracts are for internal use only.

On Thu, 2 Dec 2004, Pekka Enberg wrote:
> > I think Alexandre's definition has in fact originated from the "Design
> > by Contract" cult. And it's all pretty simple stuff:
> > 
> >   - You need to call a function in a certain way (precondition). The
> >     caller fills this part
> >     of the contract. Now the caller can _expect_ something from the
> >     function (see below).
> >   - The function does what it has promised to do and optionally
> >     returns some values
> >     (postcondition). The callee fills this part of the contract.
> >   - The function expects some parts of the system state to remain stable during
> >     execution (invariants). In the kernel, you use BUG_ON for these btw.
> > 
> > So there's nothing new here really. However, if your _tools_ support
> > Design by Contract, you can be explicit about this and enforce the
> > 'contract' during compile-time or run-time.
> > 
> > And I think you're already doing this with your "require spinlock to
> > be taken" sparse thingy...

On Thu, 2004-12-02 at 07:35 -0800, Linus Torvalds wrote:
> Yes. And I think it makes sense _internally_ for a project. 
> 
> But pretty much by definition, that's not an ABI. That's an "internal 
> interfaces" kind of thing.

And why not externally too? There are elements of a contract, I think.
The kernel, for example, dictates that you must invoke syscall A with a
pointer X pointing to an memory region of size E. Now, if the userland
follows that contract, it gets service; otherwise it won't. And as soon
as the userland _agrees_ with the contract, the kernel _must_ deliver
the service as specified by the contract.

Please note that the kernel may change the contract on its part any time
it wishes to do so. The userland can either accept the fact or reject it
and live without the service. And this is what you're essentially doing
right now. The only thing you're missing is actually _enforcing_ those
contracts.

But anyway, I think I'm twisting the original definition of Design by
Contract here for my own purposes and the only thing I violently
disagreed with you was the fact that you claimed Alexandre was making up
terms. ;)

On Thu, 2 Dec 2004, Pekka Enberg wrote:
> > I don't think it's an US thing. At the university you did your CS
> > stuff, we (well at least I) use the term 'contract' pretty much the
> > same way Alexandre does...

On Thu, 2004-12-02 at 07:35 -0800, Linus Torvalds wrote:
> It may also be a new thing. Dammit, when I started, we had USCD Pascal,
> and we were happy. None of this OO stuff ;)

Yeah, they've ruined it completely now. They're pushing Java right from
day one to youngsters :-)

			Pekka


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-02 11:30                                   ` Tomas Carnecky
@ 2004-12-02 22:17                                     ` Peter Williams
  0 siblings, 0 replies; 185+ messages in thread
From: Peter Williams @ 2004-12-02 22:17 UTC (permalink / raw)
  To: Tomas Carnecky; +Cc: Bernd Eckenfels, linux-kernel

Tomas Carnecky wrote:
> Peter Williams wrote:
> 
>> Design by contract isn't really an agreement between the caller and 
>> the callee (which may not even exist when the contract is created).  
>> It's more of a (one way) promise by the callee that if the interface 
>> is used as described in the contract that it will correctly perform 
>> the advertised operation (where the advertised operation generally 
>> includes descriptions of possible failures and how they will be 
>> signalled).  Most C APIs meet these criteria even though their pre and 
>> post conditions are expressed less formally than an Eiffel interface.
> 
> 
> Design by Contract, as seen in the Eiffel language, is not a one way 
> promise, it's a contract between the caller and callee. Both sides have
> to fullfil their part of the contract, the caller has to make sure that
> the input are valid, and (only) given that, the callee can/has to make
> sure that the caller gets the right output.
> 
> BTW, Bertrand Meyer is one of my professors, I should know the Eiffel 
> language... :)

Yes, but I still think it's essentially a one way thing.  The writer of 
the callee doesn't have to consult with the writer of the caller to 
negotiate the contract (although that option isn't ruled out either) so 
it's essentially a unilateral promise on  the part of the callee.  This 
would especially be the case when writing library interfaces as it's 
impossible for the writer of the callee to negotiate with all the 
(potential) writers of callers to agree on an interface.

> 
>> I agree but think it's important to realize that it's a unilateral 
>> promise on the part of the callee (rather than agreement between the 
>> callee and the caller) which is in accord with Linus's view of the ABI.
> 
> 
> Whenever you have two sides/parties, you have to agree on _something_, 
> otherwise you can't communicate, if you speak to someone, you have to 
> choose a language in which to speak, that's the first agreement.
> In this situation you have two sides, the kernel and the userspace. Your 
> first agreement is the syscall number, and then the arguments, the type 
> and format of the arguments, etc. Both sides have to agree on those things.

Not really.  In the final analysis, userspace has to accept what the 
kernel provides.  It's not really a conversation it's more like a 
billboard on which the kernel describes an interface and makes promises 
about how it will operate if used according to the specification (or 
contract).  If the caller fails to use the interface within the 
constraints of the specification (or contract) then the result is 
indeterminate and the callee can do whatever it likes including failing 
silently or even crashing (although that wouldn't be a good idea for a 
kernel interface).

Peter
-- 
Peter Williams                                   pwil3058@bigpond.net.au

"Learning, n. The kind of ignorance distinguishing the studious."
  -- Ambrose Bierce

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30 23:29                               ` Matt Mackall
@ 2004-12-03  1:03                                 ` Krzysztof Halasa
  0 siblings, 0 replies; 185+ messages in thread
From: Krzysztof Halasa @ 2004-12-03  1:03 UTC (permalink / raw)
  To: Matt Mackall
  Cc: Linus Torvalds, David Woodhouse, Alexandre Oliva, Paul Mackerras,
	Greg KH, Matthew Wilcox, David Howells, hch, linux-kernel,
	libc-hacker

Matt Mackall <mpm@selenic.com> writes:

> It's my opinion that it will eventually be deemed beneficial to
> separate out everything that has a legitimate reason to be used by
> userspace.

That's quite obvious and the whole thread is all about just that.

> But I'm not suggesting getting there in one go,

As this is impossible, given du -s /usr/src/linux/include/*.

> what I'm
> suggesting is how to get there incrementally. To rehash:
>
> 1. create include/user and friends
> 2. when we run across a troublesome ABI definition:
>      create include/user/foo.h and move the definition there
>      make sure include/linux/foo.h includes it
>      userspace and kernel compile as before
>      send a patch

What _is_ a "troublesome ABI definition" and who would qualify it
as such?

> 3. repeat step 2 as often as useful

Who would do that?

> 4. add new user ABI always to include/user/

Who would watch that? How do you differentiate between a user API/ABI
and an internal kernel API, if you don't know the code in question
at all?

> 5. if at some point we find that all of userspace builds from
>    include/user/ without reference to include/linux/,

How can we find that? Compiling the Fedora Code 7 would not be enough.

> 7. drop any superfluous include/linux -> include/user includes (there
>    shouldn't be many)

Why not? Kernel "private" headers can and must include ABI headers
as the private .c files do.


This won't work. include/user won't work either. There can't be any
"transition periods" nor "instant switchovers".

I can see only one way to solve the problem - gradually, without
"transition periods" etc.

1. We leave existing include/linux and include/asm (and possibly
   asm-generic etc) in place.
2. A new directory(ies) such as include/kernel (internal kernel headers)
   is created.
3. Code authors, who (I hope) know their code move the things they
   know are kernel-internal to the new directory(ies).
4. Userspace is explicitly forbidden to include anything in the new
   directory. We can even use some #ifndef __KERNEL__ #error user
   breakage as a tip.

In a year or two we can probably begin hunting remnants of the
internals in include/linux and include/asm.

We can now consider anything in include/{linux,asm} which prevents
userspace compilation etc. a bug. But bugs must be fixed by people
who know how to fix them correctly.
-- 
Krzysztof Halasa

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29  4:36           ` Jeff Garzik
  2004-11-29  4:57             ` Al Viro
  2004-11-29  7:51             ` Arjan van de Ven
@ 2004-12-05  0:49             ` Rob Landley
  2004-12-05  2:26               ` Andries Brouwer
  2 siblings, 1 reply; 185+ messages in thread
From: Rob Landley @ 2004-12-05  0:49 UTC (permalink / raw)
  To: Jeff Garzik
  Cc: Linus Torvalds, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, David Howells, hch, aoliva, linux-kernel,
	libc-hacker, Mariusz Mazur, Arjan van de Ven, andersen

On Sunday 28 November 2004 11:36 pm, Jeff Garzik wrote:
> Linus Torvalds wrote:
> > In short: having the kernel use the same names as user space is ACTIVELY
> > BAD, exactly because those names have standards-defined visibility, which
> > means that the kernel _cannot_ use them in all places anyway. So don't
> > even _try_.
> >
> > On the bigger question of what to do with kernel headers in general,
> > let's just make one thing clear: the kernel headers are for the kernel,
> > and big and painful re-organizations that don't help _existing_ user
> > space are not going to happen.
> >
> > In particular, any re-organization that breaks _existing_ uses is totally
> > pointless. If you break existing uses, you might as well _not_ re-
> > organize, since if you consider kernel headers to be purely kernel-
> > internal (like they should be, but hey, reality trumps any wishes we
> > might have), then the current organization is perfectly fine.
>
> I don't think any drastic reorganization is even necessary.
>
> Mariusz Mazur <mmazur@kernel.pl> updates
> http://ep09.pld-linux.org/~mmazur/linux-libc-headers/ for each 2.6.x
> kernel release.  linux-libc-headers are the kernel headers, with all the
> kernel-specific stuff stripped out.  i.e. userland ABI only.  Not sure
> how many distros have started picking that up yet...  I think Arjan said
> Fedora Core had, or would.

I've got a problem that mmazur's headers come down on one side of and Eric 
Andersen of the busybox project comes down on the other side of.  How the 
heck do you implement losetup without including linux/loop.h?

The way both busybox and util-linux do it is the to block copy out lots of 
ugly crap, include linux/version.h, and have #ifdefs to fix up differences 
between known kernel versions.  I'm serious.  This is from 
busybox/libbb/loop.c, but you can get an even more obfuscated variant of it 
out of util-linux:

--------------

/* Grumble...  The 2.6.x kernel breaks asm/posix_types.h
 * so we get to try and cope as best we can... */
#include <linux/version.h>
#include <asm/posix_types.h>

#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,0)
#define __bb_kernel_dev_t   __kernel_old_dev_t
#elif LINUX_VERSION_CODE < KERNEL_VERSION(2,6,0)
#define __bb_kernel_dev_t   __kernel_dev_t
#else
#define __bb_kernel_dev_t   unsigned short
#endif

/* Stuff stolen from linux/loop.h */
#define LO_NAME_SIZE        64
#define LO_KEY_SIZE         32
#define LOOP_SET_FD         0x4C00
#define LOOP_CLR_FD         0x4C01
#define LOOP_SET_STATUS     0x4C02
#define LOOP_GET_STATUS     0x4C03
struct loop_info {
    int                lo_number;
    __bb_kernel_dev_t  lo_device;
    unsigned long      lo_inode;
    __bb_kernel_dev_t  lo_rdevice;
    int                lo_offset;
    int                lo_encrypt_type;
    int                lo_encrypt_key_size;
    int                lo_flags;
    char               lo_name[LO_NAME_SIZE];
    unsigned char      lo_encrypt_key[LO_KEY_SIZE];
    unsigned long      lo_init[2];
    char               reserved[4];
};

----------------------

When I submitted a patch to clean it up on the busybox list (fixing a real 
compile problem I had with maszur's 2.6.6.0 kernel headers):
http://www.busybox.net/lists/busybox/2004-November/013128.html

I got this response:
http://www.busybox.net/lists/busybox/2004-November/013132.html

> No no no no no no no no no!
>
> Rule #1: Never ever include kernel header files in user space.
> Rule #2: See Rule #1
> Rule #3: See Rule #2

Apparently, linux/version.h isn't considered a kernel header?

Somehow, including linux/loop.h is considered bad, but including 
linux/version.h and having version specific #ifdefs in our code to fix up a 
block copied header for the specific kernel versions we know about is LESS 
incestuous knowledge.  (After all, the above will OBVIOUSLY work on BSD, is 
guaranteed to continue to compile without modification under the 2.8 kernel, 
and is in generally much cleaner than deleting all that crap and just 
including linux/loop.h.)

I still don't understand the reasoning here.  Mazur's cleaned-up kernel 
headers include a cleaned up version of linux/loop.h because loop_info simply 
isn't defined anywhere else we can #include, and it's something you need in 
order to set up loop devices.  It's a structure userspace and the kernel use 
to pass data back and forth, both need to include it.  Any _new_ header file 
created won't exist on 2.4, but this header IS there on 2.4, and 2.2, and 
2.6.  Including linux/loop.h is the only thing that will _work_, both in 
current and in future versions.

How is this _supposed_ to be done?

Rob

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-05  0:49             ` Rob Landley
@ 2004-12-05  2:26               ` Andries Brouwer
  2004-12-05 13:19                 ` David Greaves
  0 siblings, 1 reply; 185+ messages in thread
From: Andries Brouwer @ 2004-12-05  2:26 UTC (permalink / raw)
  To: Rob Landley
  Cc: Jeff Garzik, Linus Torvalds, Paul Mackerras, Greg KH,
	David Woodhouse, Matthew Wilcox, David Howells, hch, aoliva,
	linux-kernel, libc-hacker, Mariusz Mazur, Arjan van de Ven,
	andersen

On Sat, Dec 04, 2004 at 07:49:57PM -0500, Rob Landley wrote:

> How the heck do you implement losetup without including linux/loop.h?

Copy the util-linux sources.

> The way both busybox and util-linux do it is the to block copy out lots of 
> ugly crap, include linux/version.h, and have #ifdefs to fix up differences 
> between known kernel versions.  I'm serious.

Yes, a well-known problem. Util-linux has roughly

#include <linux/posix_types.h>
#include <linux/version.h>

#if LINUX_VERSION_CODE < KERNEL_VERSION(2,5,68)
#define my_dev_t __kernel_dev_t
#else
#define my_dev_t __kernel_old_dev_t
#endif

Here the struct has not changed, but the names for the types have changed.
Thus, instead of looking at <linux/version.h> and <linux/posix_types.h>
one could have a completely kernel-independent source with different defines
for each architecture.

But, all that nonsense is needed only for the obsolete struct loop_info.
Any new program should use struct loop_info64, and it has a clean definition:

struct loop_info64 {
        __u64              lo_device;                   /* ioctl r/o */
        __u64              lo_inode;                    /* ioctl r/o */
        __u64              lo_rdevice;                  /* ioctl r/o */
        __u64              lo_offset;
        __u64              lo_sizelimit;/* bytes, 0 == max available */
        __u32              lo_number;                   /* ioctl r/o */
        __u32              lo_encrypt_type;
        __u32              lo_encrypt_key_size;         /* ioctl w/o */
        __u32              lo_flags;                    /* ioctl r/o */
        __u8               lo_file_name[LO_NAME_SIZE];
        __u8               lo_crypt_name[LO_NAME_SIZE];
        __u8               lo_encrypt_key[LO_KEY_SIZE]; /* ioctl w/o */
        __u64              lo_init[2];
};

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-05  2:26               ` Andries Brouwer
@ 2004-12-05 13:19                 ` David Greaves
  2004-12-05 15:57                   ` Andries Brouwer
  0 siblings, 1 reply; 185+ messages in thread
From: David Greaves @ 2004-12-05 13:19 UTC (permalink / raw)
  To: Andries Brouwer
  Cc: Rob Landley, Jeff Garzik, Linus Torvalds, Paul Mackerras,
	Greg KH, David Woodhouse, Matthew Wilcox, David Howells, hch,
	aoliva, linux-kernel, libc-hacker, Mariusz Mazur,
	Arjan van de Ven, andersen

Err,

Isn't this WRONG.

Why is losetup considering the kernel at compile time?

Should it not be determining the kernel version at runtime and using the 
right structures then?
In which case it should have obtained a conceptual .../user/loop.h from 
a 2.4.x release and include a private copy
If 2.6.x is nice enough to provide an explicit .../user/loop.h then take 
a private copy of that into the losetup tree.

(Agreed that busybox may want to shrink as much as possible so allow 
build flags to say --with-24x-abi --with-26x-abi)

David

Andries Brouwer wrote:

>On Sat, Dec 04, 2004 at 07:49:57PM -0500, Rob Landley wrote:
>
>  
>
>>How the heck do you implement losetup without including linux/loop.h?
>>    
>>
>
>Copy the util-linux sources.
>
>  
>
>>The way both busybox and util-linux do it is the to block copy out lots of 
>>ugly crap, include linux/version.h, and have #ifdefs to fix up differences 
>>between known kernel versions.  I'm serious.
>>    
>>
>
>Yes, a well-known problem. Util-linux has roughly
>
>#include <linux/posix_types.h>
>#include <linux/version.h>
>
>#if LINUX_VERSION_CODE < KERNEL_VERSION(2,5,68)
>#define my_dev_t __kernel_dev_t
>#else
>#define my_dev_t __kernel_old_dev_t
>#endif
>
>Here the struct has not changed, but the names for the types have changed.
>Thus, instead of looking at <linux/version.h> and <linux/posix_types.h>
>one could have a completely kernel-independent source with different defines
>for each architecture.
>
>But, all that nonsense is needed only for the obsolete struct loop_info.
>Any new program should use struct loop_info64, and it has a clean definition:
>
>struct loop_info64 {
>        __u64              lo_device;                   /* ioctl r/o */
>        __u64              lo_inode;                    /* ioctl r/o */
>        __u64              lo_rdevice;                  /* ioctl r/o */
>        __u64              lo_offset;
>        __u64              lo_sizelimit;/* bytes, 0 == max available */
>        __u32              lo_number;                   /* ioctl r/o */
>        __u32              lo_encrypt_type;
>        __u32              lo_encrypt_key_size;         /* ioctl w/o */
>        __u32              lo_flags;                    /* ioctl r/o */
>        __u8               lo_file_name[LO_NAME_SIZE];
>        __u8               lo_crypt_name[LO_NAME_SIZE];
>        __u8               lo_encrypt_key[LO_KEY_SIZE]; /* ioctl w/o */
>        __u64              lo_init[2];
>};
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml/
>
>  
>


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-05 13:19                 ` David Greaves
@ 2004-12-05 15:57                   ` Andries Brouwer
  2004-12-05 23:05                     ` Rob Landley
  0 siblings, 1 reply; 185+ messages in thread
From: Andries Brouwer @ 2004-12-05 15:57 UTC (permalink / raw)
  To: David Greaves
  Cc: Andries Brouwer, Rob Landley, Jeff Garzik, Linus Torvalds,
	Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox,
	David Howells, hch, aoliva, linux-kernel, libc-hacker,
	Mariusz Mazur, Arjan van de Ven, andersen

On Sun, Dec 05, 2004 at 01:19:46PM +0000, David Greaves wrote:
> Err,
> 
> Isn't this WRONG.

You did not read my mail, or you did not understand it.

Please write the patch for losetup to avoid looking at kernel source.
If you after writing think that it is cleaner than the present setup,
submit it.

> Should it not be determining the kernel version at runtime and using the 
> right structures then?

See - you do not understand at all.
The structures do not depend on the kernel version.

Andries

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-05 15:57                   ` Andries Brouwer
@ 2004-12-05 23:05                     ` Rob Landley
  2004-12-06  9:56                       ` Andries Brouwer
  0 siblings, 1 reply; 185+ messages in thread
From: Rob Landley @ 2004-12-05 23:05 UTC (permalink / raw)
  To: Andries Brouwer
  Cc: David Greaves, Jeff Garzik, Paul Mackerras, Greg KH,
	David Woodhouse, Matthew Wilcox, David Howells, hch, aoliva,
	linux-kernel, Mariusz Mazur, Arjan van de Ven, andersen

(Trimming linus from cc: list because he asked.)

On Sunday 05 December 2004 10:57 am, Andries Brouwer wrote:
> On Sun, Dec 05, 2004 at 01:19:46PM +0000, David Greaves wrote:
> > Err,
> >
> > Isn't this WRONG.
>
> You did not read my mail, or you did not understand it.
>
> Please write the patch for losetup to avoid looking at kernel source.
> If you after writing think that it is cleaner than the present setup,
> submit it.

Avoiding looking at the kernel source for the darn structure would be 
relatively easy if the thing didn't vary by architecture.  I agree that 
hardwiring into userspace code #ifdefs for ARE_WE_ON_ALPHA is approximately 
as ugly as hardwiring in #ifdefs for IS_THIS_2.6.

Both are ugly.  This is why header files exist.  I still don't understand why 
we're not supposed to use 'em when the alternative is building incestuous 
knowledge about the kernel version we're compiling against into our 
application.

Bluntlly, I don't understand why "#include <linux/version.h>" and several 
#ifdefs is an improvement on "#include <linux/loop.h>" with no #ifdefs.

Rob

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-05 23:05                     ` Rob Landley
@ 2004-12-06  9:56                       ` Andries Brouwer
  0 siblings, 0 replies; 185+ messages in thread
From: Andries Brouwer @ 2004-12-06  9:56 UTC (permalink / raw)
  To: Rob Landley
  Cc: Andries Brouwer, David Greaves, Jeff Garzik, Paul Mackerras,
	Greg KH, David Woodhouse, Matthew Wilcox, David Howells, hch,
	aoliva, linux-kernel, Mariusz Mazur, Arjan van de Ven, andersen

On Sun, Dec 05, 2004 at 06:05:20PM -0500, Rob Landley wrote:

> Bluntlly, I don't understand why "#include <linux/version.h>" and several 
> #ifdefs is an improvement on "#include <linux/loop.h>" with no #ifdefs.

Look at the history of util-linux.

Of course it did the latter. Up to the point that broke.
Then invented a silly workaround. Up to the point that broke.
Then tried something else. Up to the point that broke.
Then gave up including this particular header and made a
private copy. Again, there is some painful history with
the private copy. I showed the current version.

You see - util-linux advertises: all kernels, all archs,
all libc versions since 4.6.27.
If the kernel improves today, it is too late for util-linux.

But yes, the include situation is painful.
It seems there is some recent progress. We'll see what happens.

Andries

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29  1:28         ` Linus Torvalds
                             ` (4 preceding siblings ...)
  2004-11-29 17:09           ` Alexandre Oliva
@ 2004-12-14  5:51           ` Werner Almesberger
  2004-12-14 15:49             ` Linus Torvalds
  5 siblings, 1 reply; 185+ messages in thread
From: Werner Almesberger @ 2004-12-14  5:51 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox,
	David Howells, hch, aoliva, linux-kernel, libc-hacker

Linus Torvalds wrote:
> Ok, this discussion has gone on for too long anyway, but let's make it
> easier for everybody. The kernel uses u8/u16/u32 because:
> 
> 	- the kernel should not depend on, or pollute user-space naming.
> 	  YOU MUST NOT USE "uint32_t" when that may not be defined, and
> 	  user-space rules for when it is defined are arcane and totally
> 	  arbitrary.

Hmm, so you're predicting problems if user space includes something
that uses uint32_t ? Wouldn't either of these work (descriptive file
names for clarity):

 1) include/linux-user/foo.h:
    #include <stdint.h>
    #include <linux-kernel-and-user-common/foo.h>

    include/linux/foo.h:
    #include <linux/stdint.h>
    #include <linux-kernel-and-user-common/foo.h>

    (And document that anything under linux-user may pull in a
    certain set of standard headers, such as stdint.h, maybe
    sys/types.h, etc.)

 2) include/linux-user/foo.h:
    #ifndef __KERNEL__
    #include <stdint.h>
    #else
    #include <linux/stdint.h>  /* if we want this */
    #endif
    ...
    foo stuff (or some include)
    ...

    Idem. (Sort of the opposite of what we have today.)

 3) just require that some stdint.h has been included before
    including linux-user/foo.h

For user space that absolutely insists on doing its own uint32_t,
a refined variant of case 3 (i.e. detailing of what exactly is
expected to be defined) or, in case 1, direct use of
linux-kernel-and-user-common/foo.h along with the definition from
case 3 should work.

There are other cases where a header file may pull in other
definitions, e.g. for fcntl.h, POSIX decrees

| Inclusion of the <fcntl.h> header may also make visible all
| symbols from <sys/stat.h> and <unistd.h>.

So the semantics of for cases 1 and 2 would be consistent with
current POSIX practice.

What am I missing ?

- Werner

-- 
  _________________________________________________________________________
 / Werner Almesberger, Buenos Aires, Argentina         wa@almesberger.net /
/_http://www.almesberger.net/____________________________________________/

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-29 20:01 ` H. Peter Anvin
  2004-11-30  0:34   ` Kyle Moffett
@ 2004-12-14  7:07   ` Werner Almesberger
  1 sibling, 0 replies; 185+ messages in thread
From: Werner Almesberger @ 2004-12-14  7:07 UTC (permalink / raw)
  To: H. Peter Anvin; +Cc: linux-kernel

H. Peter Anvin wrote:
> Since we can't use typedefs on
> struct tags, I suggest:

Wouldn't this be a wonderful extension to ask from the gcc folks ?
typedef mumble struct foo; /* or union, enum */
with "mumble" either a "struct bar" or a typedef thereof.

- Werner

-- 
  _________________________________________________________________________
 / Werner Almesberger, Buenos Aires, Argentina         wa@almesberger.net /
/_http://www.almesberger.net/____________________________________________/

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-14  5:51           ` Werner Almesberger
@ 2004-12-14 15:49             ` Linus Torvalds
  2004-12-14 16:50               ` Werner Almesberger
  2004-12-14 16:55               ` Andreas Schwab
  0 siblings, 2 replies; 185+ messages in thread
From: Linus Torvalds @ 2004-12-14 15:49 UTC (permalink / raw)
  To: Werner Almesberger
  Cc: Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox,
	David Howells, hch, aoliva, linux-kernel, libc-hacker



On Tue, 14 Dec 2004, Werner Almesberger wrote:
> 
> Hmm, so you're predicting problems if user space includes something
> that uses uint32_t ? Wouldn't either of these work (descriptive file
> names for clarity):
> 
>  1) include/linux-user/foo.h:
>     #include <stdint.h>
>     #include <linux-kernel-and-user-common/foo.h>

No. Because when you include <sys/ioctl.h> (which includes the 
<linux-user/foo.h>, the POSIX/SuS/whatever namespace rules say that YOU 
MUST NOT pollute the namespace with the names from stdint.h.

The user is supposed to see "int32_t" and friends _only_ if the user 
himself includes <stdint.h> or one of the very specific headers that is 
documented by the standard to include it.

Trust me. We are NOT going to use <stdint.h> in the kernel. 

This is a common issue with namespace pollution. For example, this program 
is perfectly valid afaik (well, except for being _stupid_, but that's 
another issue):

	#include <stdio.h>

	const char *int32_t(int i)
	{
		return i ? "non-zero" : "zero";
	}

	int main(int argc, char **argv)
	{
		return printf("argc is %s\n", int32_t(argc));
	}

Get the picture? The user may _use_ names that are defined for the 
standard - like "strlen()" or "uint32_t" - for his own nefarious purposes, 
because they are _only_ defined if you include the right header files.

And trust me, the rules are really arcane. Not only do you have several 
standards, and several versions, you have various local rules too, ie gcc 
and glibc make up their own rules about things that depend on compiler 
flags etc. 

And that is why a kernel header file must not define them, and by 
implication, cannot use these names.

Because the whole _point_ of having kernel header files that can be used 
in user-space is that they are helpers for things like <sys/ioctl.h> etc, 
that would just copy them.

If we are going to pollute the user-visible namespace in random ways, the
whole discussion is pointless. At that point, glibc (or any other libc)  
can't use those API headers anyway.

Remember: the _biggest_ reason to make kernel headers available is not to 
user programs that want them, but to libc and friends.

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-14 15:49             ` Linus Torvalds
@ 2004-12-14 16:50               ` Werner Almesberger
  2004-12-14 17:58                 ` Linus Torvalds
  2004-12-14 19:23                 ` Horst von Brand
  2004-12-14 16:55               ` Andreas Schwab
  1 sibling, 2 replies; 185+ messages in thread
From: Werner Almesberger @ 2004-12-14 16:50 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox,
	David Howells, hch, aoliva, linux-kernel, libc-hacker

Linus Torvalds wrote:
> No. Because when you include <sys/ioctl.h> (which includes the 
> <linux-user/foo.h>, the POSIX/SuS/whatever namespace rules say that YOU 
> MUST NOT pollute the namespace with the names from stdint.h.

Hmm, the POSIX 1003.1 2004 edition (which includes SuS) wisely
remains silent on the issue of sys/ioctl.h, but I see what you
mean.

I think some of my confusion arises from the fact that there aren't
many well-known interfaces that use uint32_t and friends to start
with. Instead, they use ssize_t, off_t and such (which admittedly
have similar problems, e.g. in cases where one can't simply get all
of sys/types.h), or private inventions like u_int32_t and such.

Therefore, stdint.h types would mainly be used with new interfaces,
or in intermediate definitions which are not themselves part of the
interface. Of course, the latter would have to consider pollution
issues.

Would you agree that the use of stdint.h types is acceptable for
new interfaces, provided that these come with the warning that
their use may pull in all of stdint.h ?

> And trust me, the rules are really arcane. Not only do you have several 
> standards, and several versions, you have various local rules too, ie gcc 
> and glibc make up their own rules about things that depend on compiler 
> flags etc. 

If this is as unpredictable as you describe it, it would mean
that also new interfaces which need to specify an exact integer
size would require new sets of type names. So horrors like
my_uint32_t, project-specific and of course conflicting
definitions of ULONG (really really meaning 32 bit, at least
sometimes), etc. would still be with us for a long time.

Let me add that I've happily used the standard integer names for
a while, inside and outside of the kernel, so far without ill
effects. Maybe I've just been lucky.

> Remember: the _biggest_ reason to make kernel headers available is not to 
> user programs that want them, but to libc and friends.

Okay, for me it's usually exactly the opposite :-) New tools
that need to share fairly private interfaces with the kernel.

Thanks,
- Werner

-- 
  _________________________________________________________________________
 / Werner Almesberger, Buenos Aires, Argentina         wa@almesberger.net /
/_http://www.almesberger.net/____________________________________________/

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-14 15:49             ` Linus Torvalds
  2004-12-14 16:50               ` Werner Almesberger
@ 2004-12-14 16:55               ` Andreas Schwab
  2004-12-14 17:59                 ` Linus Torvalds
  1 sibling, 1 reply; 185+ messages in thread
From: Andreas Schwab @ 2004-12-14 16:55 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Werner Almesberger, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, David Howells, hch, aoliva, linux-kernel,
	libc-hacker

Linus Torvalds <torvalds@osdl.org> writes:

> This is a common issue with namespace pollution. For example, this program 
> is perfectly valid afaik (well, except for being _stupid_, but that's 
> another issue):
>
> 	#include <stdio.h>
>
> 	const char *int32_t(int i)
> 	{
> 		return i ? "non-zero" : "zero";
> 	}

Actually this is not allowed in POSIX.  _Any_ header may define any
identifier ending with "_t".

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-14 16:50               ` Werner Almesberger
@ 2004-12-14 17:58                 ` Linus Torvalds
  2004-12-14 21:46                   ` Werner Almesberger
  2004-12-14 19:23                 ` Horst von Brand
  1 sibling, 1 reply; 185+ messages in thread
From: Linus Torvalds @ 2004-12-14 17:58 UTC (permalink / raw)
  To: Werner Almesberger
  Cc: Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox,
	David Howells, hch, aoliva, linux-kernel, libc-hacker



On Tue, 14 Dec 2004, Werner Almesberger wrote:
> 
> Would you agree that the use of stdint.h types is acceptable for
> new interfaces, provided that these come with the warning that
> their use may pull in all of stdint.h ?

Not really. 

Two reasons:
 - The fact is, things are interdependent. And this thing is all about 
   _avoiding_ implicit dependencies, and rules like "if you include _this_ 
   particular header file, then you must include yyy.h first, or you can't 
   do zzzz".
 - uint32_t and friends aren't all that nice anyway. Quite frankly, there 
   is nada, zilch, _zero_, reason to use them over the __u8/__u16/__u32.
   In contrast, the __u8/__u16/__u32 types have obvious advantages, 
   ranging from the _lack_ of namespace collission issue to just plain 
   consistency.

iow, using "uint32_t" and friends is _stupid_. The only fathomable 
argument for doing so would be consistency, but since BY DEFINITION we 
cannot be consistent if we use them (since we cannot and must not use them 
anywhere), that _single_ reason for using them is actually the biggest 
reason to NOT use them.

See my point?

> If this is as unpredictable as you describe it, it would mean
> that also new interfaces which need to specify an exact integer
> size would require new sets of type names.

No. They require an OLD set of type names, that have existed for years in
the kernel, and that are already used widely.

In other words, they require __u8/__u16/__u32/__u64. That _is_ the way to 
specify a sized type in kernel headers. It has been that way for years, 
it's extremely readable, it's consistent, and it's compact.

In other words, u8/u16/u32/u64 (and the signed versions: s8 and friends)  
_are_ the standard names in the kernel, and the __ versions have always 
existed alongside them for things like header files that have namespace 
issues.

So forget about that stupid abortion called "uint32_t" already. It buys
you absolutely nothing.

			Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-14 16:55               ` Andreas Schwab
@ 2004-12-14 17:59                 ` Linus Torvalds
  0 siblings, 0 replies; 185+ messages in thread
From: Linus Torvalds @ 2004-12-14 17:59 UTC (permalink / raw)
  To: Andreas Schwab
  Cc: Werner Almesberger, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, David Howells, hch, aoliva, linux-kernel,
	libc-hacker



On Tue, 14 Dec 2004, Andreas Schwab wrote:
>
> Linus Torvalds <torvalds@osdl.org> writes:
> 
> > This is a common issue with namespace pollution. For example, this program 
> > is perfectly valid afaik (well, except for being _stupid_, but that's 
> > another issue):
> >
> > 	#include <stdio.h>
> >
> > 	const char *int32_t(int i)
> > 	{
> > 		return i ? "non-zero" : "zero";
> > 	}
> 
> Actually this is not allowed in POSIX.  _Any_ header may define any
> identifier ending with "_t".

.. and that's POSIX.

It's not true for BSD_SOURCE, for example.

WHICH IS MY POINT, DAMMIT!

How hard is it to understand? There are millions of different standards, 
many of them without any standards body at all, and just plain standards 
by virtue of "that's how people have done it for years".

uint32_t is totally useless for the kernel. We should never ever use it.

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__ 
  2004-12-14 16:50               ` Werner Almesberger
  2004-12-14 17:58                 ` Linus Torvalds
@ 2004-12-14 19:23                 ` Horst von Brand
  2004-12-14 19:45                   ` Sam Ravnborg
  1 sibling, 1 reply; 185+ messages in thread
From: Horst von Brand @ 2004-12-14 19:23 UTC (permalink / raw)
  To: Werner Almesberger
  Cc: Linus Torvalds, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, David Howells, hch, aoliva, linux-kernel,
	libc-hacker

Werner Almesberger <wa@almesberger.net> said:
> Linus Torvalds wrote:
> > No. Because when you include <sys/ioctl.h> (which includes the 
> > <linux-user/foo.h>, the POSIX/SuS/whatever namespace rules say that YOU 
> > MUST NOT pollute the namespace with the names from stdint.h.

[...]

> Therefore, stdint.h types would mainly be used with new interfaces,
> or in intermediate definitions which are not themselves part of the
> interface. Of course, the latter would have to consider pollution
> issues.

They _can't_ show up in interfaces unless you specify that stdint.h has to
be included before them... and I'd awise against any needless interface
fattening. Besides, using them in the kernel would then mean pulling in
stdint.h there... and you get the same mess, the other way around ("What
userspace headers are OK to pull in when compiling the kernel?"). Better
don't.

[...]

> > And trust me, the rules are really arcane. Not only do you have several 
> > standards, and several versions, you have various local rules too, ie gcc 
> > and glibc make up their own rules about things that depend on compiler 
> > flags etc. 

> If this is as unpredictable as you describe it, it would mean that also
> new interfaces which need to specify an exact integer size would require
> new sets of type names.

That's what all the _u8 and such are. Note that any name starting with _ or
__ is explicitly reserved by the standard (i.e., users should never use
them), so they are (relatively) safe.

>                         So horrors like my_uint32_t, project-specific and
> of course conflicting definitions of ULONG (really really meaning 32 bit,
> at least sometimes), etc. would still be with us for a long time.

Better use stdint.h where possible.

> Let me add that I've happily used the standard integer names for
> a while, inside and outside of the kernel, so far without ill
> effects. Maybe I've just been lucky.

No. You just haven't commited horrors like the ones Linus showed. No
halfway sane C programmer would, but they are quite legal (and must work).

> > Remember: the _biggest_ reason to make kernel headers available is not to 
> > user programs that want them, but to libc and friends.

> Okay, for me it's usually exactly the opposite :-) New tools
> that need to share fairly private interfaces with the kernel.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-14 19:23                 ` Horst von Brand
@ 2004-12-14 19:45                   ` Sam Ravnborg
  2004-12-14 19:58                     ` Linus Torvalds
  0 siblings, 1 reply; 185+ messages in thread
From: Sam Ravnborg @ 2004-12-14 19:45 UTC (permalink / raw)
  To: Horst von Brand
  Cc: Werner Almesberger, Linus Torvalds, Paul Mackerras, Greg KH,
	David Woodhouse, Matthew Wilcox, David Howells, hch, aoliva,
	linux-kernel, libc-hacker

On Tue, Dec 14, 2004 at 04:23:12PM -0300, Horst von Brand wrote:
> 
> > Therefore, stdint.h types would mainly be used with new interfaces,
> > or in intermediate definitions which are not themselves part of the
> > interface. Of course, the latter would have to consider pollution
> > issues.
> 
> They _can't_ show up in interfaces unless you specify that stdint.h has to
> be included before them... and I'd awise against any needless interface
> fattening. Besides, using them in the kernel would then mean pulling in
> stdint.h there... and you get the same mess, the other way around ("What
> userspace headers are OK to pull in when compiling the kernel?"). Better
> don't.

Today we only pull in stdarg.h from userspace - actually a compiler
dependent file. We shall not introduce anything to pull more stuff in
from userspace. We have namespace issues enough - keeping the kernel out
of userspace namespace is important.

	Sam

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-14 19:45                   ` Sam Ravnborg
@ 2004-12-14 19:58                     ` Linus Torvalds
  2004-12-14 20:25                       ` Andreas Schwab
  0 siblings, 1 reply; 185+ messages in thread
From: Linus Torvalds @ 2004-12-14 19:58 UTC (permalink / raw)
  To: Sam Ravnborg
  Cc: Horst von Brand, Werner Almesberger, Paul Mackerras, Greg KH,
	David Woodhouse, Matthew Wilcox, David Howells, hch, aoliva,
	linux-kernel, libc-hacker



On Tue, 14 Dec 2004, Sam Ravnborg wrote:
> 
> Today we only pull in stdarg.h from userspace - actually a compiler
> dependent file.

Indeed. You'll notice that gcc doesn't even put stdarg.h in /usr/include, 
it's in the compiler-specific header file directory, usually something 
like /usr/lib/gcc-lib/<vendor>/<version>/include.

We used to compile with "-nostdinc" to make sure that you couldn't include 
user files even by mistake, but that was removed for some reason I can't 
for the life of me remember any more.

We probably should do it again. Something vaguely like

	CFLAGS += -nostdinc -I $(CC -print-file-name=include)

should do it (and still pick up "stdarg.h").

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-14 19:58                     ` Linus Torvalds
@ 2004-12-14 20:25                       ` Andreas Schwab
  2004-12-14 20:33                         ` Linus Torvalds
  0 siblings, 1 reply; 185+ messages in thread
From: Andreas Schwab @ 2004-12-14 20:25 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Sam Ravnborg, Horst von Brand, Werner Almesberger,
	Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox,
	David Howells, hch, aoliva, linux-kernel, libc-hacker

Linus Torvalds <torvalds@osdl.org> writes:

> We used to compile with "-nostdinc" to make sure that you couldn't include 
> user files even by mistake, but that was removed for some reason I can't 
> for the life of me remember any more.

??? We still do that, see NOSTDINC_FLAGS.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-14 20:25                       ` Andreas Schwab
@ 2004-12-14 20:33                         ` Linus Torvalds
  0 siblings, 0 replies; 185+ messages in thread
From: Linus Torvalds @ 2004-12-14 20:33 UTC (permalink / raw)
  To: Andreas Schwab
  Cc: Sam Ravnborg, Horst von Brand, Werner Almesberger,
	Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox,
	David Howells, hch, aoliva, linux-kernel, libc-hacker



On Tue, 14 Dec 2004, Andreas Schwab wrote:
> 
> ??? We still do that, see NOSTDINC_FLAGS.

Ahh, yup, there is it. Goodie, I was just confused.

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-14 17:58                 ` Linus Torvalds
@ 2004-12-14 21:46                   ` Werner Almesberger
  2004-12-14 21:58                     ` Linus Torvalds
  2004-12-14 23:49                     ` Krzysztof Halasa
  0 siblings, 2 replies; 185+ messages in thread
From: Werner Almesberger @ 2004-12-14 21:46 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox,
	David Howells, hch, aoliva, linux-kernel, libc-hacker

Linus Torvalds wrote:
> iow, using "uint32_t" and friends is _stupid_. The only fathomable 
> argument for doing so would be consistency,

I think consistency is a worthwhile goal in this case. I guess we
can work towards consistency even without using uint32_t in kernel
headers.

This still bears the risk that people will copy __u32 and friends
from headers to then non-portable applications, but I guess that's
something we just have live with.

Since C now finally has standard types with well-defined sizes, I
think there's little reason for applications not to use these types.
So peer pressure should eventually make everyone at least compatible.

A first step towards enabling converence would be to _guarantee_
that glibc types are _identical_ to the kernel types, e.g.

/usr/include/stdint.h:
#include <linux-user/types.h> /* only __u32 and such */

typedef __u32 uint32_t;
...

(If they just "happen to be the same", people may still not trust
this, and will feel justified inventing their own schemes. And as
ugly as uint32_t may look, it's still infinitely better than code
where each library contributes its own set of sized integers ...)

Then, in a few years, when uint32_t and friends are considered
as much a part of the C language as "int", we might be able to
sed s/__u32/uint32_t/g the kernel, and obtain perfect consistency.

Does this sound reasonable ?

> So forget about that stupid abortion called "uint32_t" already. It buys
> you absolutely nothing.

Now, what do we do with them when they are inside the kernel,
far from any interfaces ? Live and let live ? Deprecate them
early on January 1st, when nobody is watching ? :-)

Thanks,
- Werner

-- 
  _________________________________________________________________________
 / Werner Almesberger, Buenos Aires, Argentina         wa@almesberger.net /
/_http://www.almesberger.net/____________________________________________/

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-14 21:46                   ` Werner Almesberger
@ 2004-12-14 21:58                     ` Linus Torvalds
  2004-12-14 23:49                     ` Krzysztof Halasa
  1 sibling, 0 replies; 185+ messages in thread
From: Linus Torvalds @ 2004-12-14 21:58 UTC (permalink / raw)
  To: Werner Almesberger
  Cc: Paul Mackerras, Greg KH, David Woodhouse, Matthew Wilcox,
	David Howells, hch, aoliva, linux-kernel, libc-hacker



On Tue, 14 Dec 2004, Werner Almesberger wrote:
> > So forget about that stupid abortion called "uint32_t" already. It buys
> > you absolutely nothing.
> 
> Now, what do we do with them when they are inside the kernel,
> far from any interfaces ? Live and let live ?

Yes. As long as they don't cause problems, I'm a big fan of "let
developers make their own choices". I think that uint32_t and friends are
totally useless in the kernel, but while I have strong opinions, I try to
not force those opinions on others too much.

Quite frankly, it's unlikely to cause problems in practice. Having strong 
opinions and enjoying the occasional flame war is one thing. Trying to 
rail against other people who don't share those opinions and force them to 
change their code is a totally different thing.

[ It has absolutely nothing to do with me being lazy. Oh, no. I'm not 
  lazy. Much better to attribute it to some lofty goal like "live and let 
  live". If the two just _happen_ to co-incide, all the better ;-]

			Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-14 21:46                   ` Werner Almesberger
  2004-12-14 21:58                     ` Linus Torvalds
@ 2004-12-14 23:49                     ` Krzysztof Halasa
  2004-12-15  0:09                       ` Werner Almesberger
  1 sibling, 1 reply; 185+ messages in thread
From: Krzysztof Halasa @ 2004-12-14 23:49 UTC (permalink / raw)
  To: Werner Almesberger
  Cc: Linus Torvalds, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, David Howells, hch, aoliva, linux-kernel,
	libc-hacker

Werner Almesberger <wa@almesberger.net> writes:

> A first step towards enabling converence would be to _guarantee_
> that glibc types are _identical_ to the kernel types, e.g.
>
> /usr/include/stdint.h:
> #include <linux-user/types.h> /* only __u32 and such */
>
> typedef __u32 uint32_t;
> ...
>
> (If they just "happen to be the same", people may still not trust
> this, and will feel justified inventing their own schemes. And as
> ugly as uint32_t may look, it's still infinitely better than code
> where each library contributes its own set of sized integers ...)

Not sure about this. I think such types should be defined by the
kernel, for it's use only. The libc (and any other userspace
software) may probably use them (if they have to), but must not
(re)define them.
The userspace should probably best use standard types. This is
userspace to know if their uint32_t has standard meaning equivalent
to __u32 or if it's actually a function name.


So i think we should have:

kernel-include/headers.h:
#define __u32 ...

struct some_structure {
       __u32 var;
}str1;


and then userspace.c (libc et al) should:

#include linux/headers.h

str1 v;
uint32_t variable = v.var;

or maybe with some type conversion:

long long v2 = v.var;
unsigned char v3 = v.var;
-- 
Krzysztof Halasa

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-14 23:49                     ` Krzysztof Halasa
@ 2004-12-15  0:09                       ` Werner Almesberger
  2004-12-16  0:58                         ` Krzysztof Halasa
  0 siblings, 1 reply; 185+ messages in thread
From: Werner Almesberger @ 2004-12-15  0:09 UTC (permalink / raw)
  To: Krzysztof Halasa
  Cc: Linus Torvalds, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, David Howells, hch, aoliva, linux-kernel,
	libc-hacker

Krzysztof Halasa wrote:
> Not sure about this. I think such types should be defined by the
> kernel, for it's use only. The libc (and any other userspace
> software) may probably use them (if they have to), but must not
> (re)define them.

That's not what should be happening there: if the kernel only uses
__u32 and friends but never uint32_t et al. at an interface,

typedef __u32 uint32_t;

should be a perfectly safe thing for glibc's stdint.h to do.

> The userspace should probably best use standard types.

Yes.

> This is
> userspace to know if their uint32_t has standard meaning equivalent
> to __u32 or if it's actually a function name.

Yes, but that's decided by whether you include stdint.h or
anything that has the effect of including stdint.h. What I'm
saying is that there should be a guarantee that __u32 and
uint32_t are always identical types. If they are merely
assignment-compatible, you run into problems like this

/* somewhere in a kernel-to-glibc interface (on 32 bit) */
typedef unsigned int __u32;
...
static __u32 foo;

/* elsewhere, in glibc (on 32 bit) */
typedef unsigned long uint32_t;
...
static uint32_t bar;

/* finally, in the application */
if (&foo == &bar)
	/* do something */;

warning: comparison of distinct pointer types lacks a cast

> and then userspace.c (libc et al) should:
> 
> #include linux/headers.h
> 
> str1 v;
> uint32_t variable = v.var;

Interfaces are normally specified with the explicit data type of
each item, in particular struct members. The masses will probably
look for solid branches instead of olive twigs for you if you try
to break this kind of assurance :-)

- Werner

-- 
  _________________________________________________________________________
 / Werner Almesberger, Buenos Aires, Argentina         wa@almesberger.net /
/_http://www.almesberger.net/____________________________________________/

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-12-15  0:09                       ` Werner Almesberger
@ 2004-12-16  0:58                         ` Krzysztof Halasa
  0 siblings, 0 replies; 185+ messages in thread
From: Krzysztof Halasa @ 2004-12-16  0:58 UTC (permalink / raw)
  To: Werner Almesberger
  Cc: Linus Torvalds, Paul Mackerras, Greg KH, David Woodhouse,
	Matthew Wilcox, David Howells, hch, aoliva, linux-kernel,
	libc-hacker

Werner Almesberger <wa@almesberger.net> writes:

> That's not what should be happening there: if the kernel only uses
> __u32 and friends but never uint32_t et al. at an interface,
>
> typedef __u32 uint32_t;
>
> should be a perfectly safe thing for glibc's stdint.h to do.

Aaah, right. I thought about #define there, it would be the other way
around and thus wrong. Looks like I have to read my mail a bit earlier.
-- 
Krzysztof Halasa

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30  4:22 ` Linus Torvalds
@ 2004-11-30  6:51   ` H. Peter Anvin
  0 siblings, 0 replies; 185+ messages in thread
From: H. Peter Anvin @ 2004-11-30  6:51 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: jt, Linux kernel mailing list, David Howells, Jeff Garzik, Mariusz Mazur

Linus Torvalds wrote:
> 
> On Mon, 29 Nov 2004, Jean Tourrilhes wrote:
> 
>>	So, which kernel ABI should be present on my system in
>>/usr/include/linux and /usr/include/asm ? Should I use the ABI from
>>2.6.X, 2.4.X or 2.2.X ?
> 
> 
> I have always felt (pretty strongly) that the /usr/include/xxx contents 
> should not be kernel-dependent, but be linked to your glibc version. 
> That's why the symlink from /usr/include/xxx to /usr/src/linux/include/
> has been deprecated for the last, oh about ten years now..
> 
> Yes, there are some _very_ specific things which might care about system 
> calls or ioctl's that have been added later, but let's face it, we don't 
> actually do that very often. The kernel may change at a rapid pace, but 
> user interfaces don't, and user interfaces that would bypass the C library 
> change even less frequently.
> 

More to the point, though, is that they should be supersets of each 
other, which means you should be able to upgrade them to the latest version.

Most of the ABI changes, after all, are new structures needed for 
various things.

	-hpa


^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-30  1:43 Jean Tourrilhes
@ 2004-11-30  4:22 ` Linus Torvalds
  2004-11-30  6:51   ` H. Peter Anvin
  0 siblings, 1 reply; 185+ messages in thread
From: Linus Torvalds @ 2004-11-30  4:22 UTC (permalink / raw)
  To: jt
  Cc: Linux kernel mailing list, David Howells, Jeff Garzik,
	Mariusz Mazur, H. Peter Anvin



On Mon, 29 Nov 2004, Jean Tourrilhes wrote:
>
> 	So, which kernel ABI should be present on my system in
> /usr/include/linux and /usr/include/asm ? Should I use the ABI from
> 2.6.X, 2.4.X or 2.2.X ?

I have always felt (pretty strongly) that the /usr/include/xxx contents 
should not be kernel-dependent, but be linked to your glibc version. 
That's why the symlink from /usr/include/xxx to /usr/src/linux/include/
has been deprecated for the last, oh about ten years now..

Yes, there are some _very_ specific things which might care about system 
calls or ioctl's that have been added later, but let's face it, we don't 
actually do that very often. The kernel may change at a rapid pace, but 
user interfaces don't, and user interfaces that would bypass the C library 
change even less frequently.

		Linus

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
@ 2004-11-30  1:43 Jean Tourrilhes
  2004-11-30  4:22 ` Linus Torvalds
  0 siblings, 1 reply; 185+ messages in thread
From: Jean Tourrilhes @ 2004-11-30  1:43 UTC (permalink / raw)
  To: Linux kernel mailing list, David Howells, Jeff Garzik,
	Linus Torvalds, Mariusz Mazur, H. Peter Anvin

David Howells wrote :
> We've been discussing splitting the kernel headers into userspace API headers
> and kernel internal headers and deprecating the __KERNEL__ macro.

Linus Torvalds wrote :
> On Mon, 29 Nov 2004, Alexandre Oliva wrote:
> >
> > - Linux gets to define the ABI between kernel and userland, and
> >   userland must duplicate the contents of headers in which the kernel
> >   defines the kernel<->userland ABI, tracking changes in them in the
> >   hope that nothing falls through the cracks
> 
> This is unquestionably true. The kernel obviously _does_ define the ABI, 
> and userland just lives with it. At some point you have to track things, 
> just because new features etc just can't be sanely handled any other way. 
> 
> That said, I think we can make the tracking _easier_. 

	Nice discussion, but nobody is discussing my main concern,
even if Linus come close to it.
	On my box, I do have multiple kernels. Some 2.4.X, some 2.6.X,
I may even find some 2.2.X on some of my boxes. Depending on what I
do, and the bug reports send to me, I boot one of these kernel or
another. Because Linus is so productive, I update my kernel
frequently, without changing the user space. I know that some
distributions are also offering various kernel versions, and let the
user choose.
	So, which kernel ABI should be present on my system in
/usr/include/linux and /usr/include/asm ? Should I use the ABI from
2.6.X, 2.4.X or 2.2.X ? Should I take the time to create a sanitised
version of the kernel header every time I install a new kernel, even
though I may revert back to the previous kernel ? Should I recompile
applications using kernel headers at each kernel upgrade ?
	Currently, it doesn't matter, as so few applications are using
those headers, and most changes are highly backward compatible. And
the application that are deeply intimate with the kernel probably do
run-time tracking of features already, so don't really need those
headers...

	However, I believe we need to at least ask this question, even
if I don't think we will come up with a good answer...

	Have fun...

	Jean

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-28  0:55       ` ak
@ 2004-11-28 13:32         ` Arnd Bergmann
  0 siblings, 0 replies; 185+ messages in thread
From: Arnd Bergmann @ 2004-11-28 13:32 UTC (permalink / raw)
  To: ak
  Cc: linux-kernel, Sam Ravnborg, David Howells, torvalds, hch,
	matthew, dwmw2, aoliva, Andreas Steinmetz


[-- Attachment #1: Type: text/plain, Size: 1264 bytes --]

On Sünndag 28 November 2004 01:55, ak@muc.de wrote:
> Please don't do that. x86_64 are not really synchronized in development
> (unlike s390/s390x) and I don't really want too coordinate too much
> with the i386 people when I change something in asm. There are also
> significant differences in some places already.
> Keep it separate is imho far better.
> 
The ABI is actually a very small part of the asm/ files [1] and typically
one that does not see many changes. I found 5 files (fcntl.h ioctl.h
siginfo.h statfs.h types.h) that have trivial differences between i386
and x86_64 and can be merged, as well as 6 files (posix_types.h ptrace.h
sigcontext.h signal.h stat.h unistd.h) that are so different that it
makes sense to keep them completely separate files with an extra file
to choose between them.

The rest is either not part of the ABI or already identical.

	Arnd <><

[1] A quick browse over the files suggests we need only:
  errno.h fcntl.h ioctl.h ioctls.h ipc.h ipcbuf.h msgbuf.h
  param.h poll.h posix_types.h ptrace.h resource.h sembuf.h
  shmbuf.h shmparam.h sigcontext.h siginfo.h signal.h
  socket.h sockios.h stat.h statfs.h termbits.h termios.h
  types.h unistd.h

  If anyone has a better list, please post it.

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
       [not found]     ` <35kb6-46Q-25@gated-at.bofh.it>
@ 2004-11-28  0:55       ` ak
  2004-11-28 13:32         ` Arnd Bergmann
  0 siblings, 1 reply; 185+ messages in thread
From: ak @ 2004-11-28  0:55 UTC (permalink / raw)
  To: Arnd Bergmann; +Cc: linux-kernel

> I think we can get rid of this hack when we move to split kernel headers.
> parisc, s390 and mips already have combined headers, and it should not be
> too hard to combine the user ABI headers for sparc, ppc and x86_64 as well,
> without having to merge the complete architecture and kernel header trees
> for them.

Please don't do that. x86_64 are not really synchronized in development
(unlike s390/s390x) and I don't really want too coordinate too much
with the i386 people when I change something in asm. There are also
significant differences in some places already.
Keep it separate is imho far better.

-Andi

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
  2004-11-27 19:43 Dan Kegel
@ 2004-11-27 21:06 ` Sam Ravnborg
  0 siblings, 0 replies; 185+ messages in thread
From: Sam Ravnborg @ 2004-11-27 21:06 UTC (permalink / raw)
  To: Dan Kegel; +Cc: Linux Kernel Mailing List

On Sat, Nov 27, 2004 at 11:43:06AM -0800, Dan Kegel wrote:
> >
> >for me i would install all those kernel realted files into 
> >the well known /lib/modules/<kernel-version>. 
> 
> IMHO the script should let you install the headers
> wherever you like.  In particular, in crosstool,
> I would like to install the headers somewhere like
> /opt/crosstool/$TARGET/include rather than /usr/include.

Agreed and trivial to do.
One should recall that for current kernel the following path is valid:
/lib/modules/`uname -r`/source/include/*

This is valid for 2.6.7 or something.

	Sam

^ permalink raw reply	[flat|nested] 185+ messages in thread

* Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
@ 2004-11-27 19:43 Dan Kegel
  2004-11-27 21:06 ` Sam Ravnborg
  0 siblings, 1 reply; 185+ messages in thread
From: Dan Kegel @ 2004-11-27 19:43 UTC (permalink / raw)
  To: Linux Kernel Mailing List

Alexander Stohr wrote:
>> Matthew Wilcox wrote:
>> > Indeed.  We could also make this transparent to userspace by using a script
>> > to copy the user-* headers to /usr/include.  Something like this:
> 
> some administrator would like to only create symlinks for saving disk space.
> 
> others would like a true "install" with copying files so that they can delete
> the kernel sources anytime they do want.
> 
> for me i would install all those kernel realted files into 
> the well known /lib/modules/<kernel-version>. 

IMHO the script should let you install the headers
wherever you like.  In particular, in crosstool,
I would like to install the headers somewhere like
/opt/crosstool/$TARGET/include rather than /usr/include.
- Dan

-- 
Trying to get a job as a c++ developer?  See http://kegel.com/academy/getting-hired.html

^ permalink raw reply	[flat|nested] 185+ messages in thread

end of thread, back to index

Thread overview: 185+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2004-11-25 15:13 [RFC] Splitting kernel headers and deprecating __KERNEL__ David Howells
2004-11-25 15:23 ` David Woodhouse
2004-11-25 16:54 ` Matthew Wilcox
2004-11-25 18:17   ` David Woodhouse
2004-11-27  3:24     ` Greg KH
2004-11-28 23:37       ` Paul Mackerras
2004-11-29  1:28         ` Linus Torvalds
2004-11-29  4:36           ` Jeff Garzik
2004-11-29  4:57             ` Al Viro
2004-11-29  5:27               ` Tim Hockin
2004-11-29  7:51             ` Arjan van de Ven
2004-12-05  0:49             ` Rob Landley
2004-12-05  2:26               ` Andries Brouwer
2004-12-05 13:19                 ` David Greaves
2004-12-05 15:57                   ` Andries Brouwer
2004-12-05 23:05                     ` Rob Landley
2004-12-06  9:56                       ` Andries Brouwer
2004-11-29  9:42           ` David Woodhouse
2004-11-30  8:08             ` Alex Riesen
2004-11-30  8:57               ` Alex Riesen
2004-12-01 11:46             ` Ralf Baechle
2004-12-02  1:21               ` H. Peter Anvin
2004-11-29  9:53           ` Paul Mackerras
2004-11-29  9:57             ` Arjan van de Ven
2004-11-29 10:01               ` David Woodhouse
2004-11-29 11:41           ` Mariusz Mazur
2004-11-29 17:09           ` Alexandre Oliva
2004-11-29 17:41             ` Linus Torvalds
2004-11-29 22:44               ` Al Viro
2004-12-01 11:36                 ` Roman Zippel
     [not found]               ` <oract0thnj.fsf@livre.redhat.lsd.ic.unicamp.br>
2004-11-29 23:00                 ` Linus Torvalds
2004-11-30  0:45                   ` H. Peter Anvin
2004-11-30  5:31                   ` Herbert Poetzl
2004-11-30  6:44                   ` bert hubert
2004-11-30 21:22                   ` Alexandre Oliva
2004-11-30 22:22                     ` Linus Torvalds
2004-11-30 22:57                       ` Alexandre Oliva
2004-11-30 23:12                         ` Linus Torvalds
2004-12-01 19:41                           ` Alexandre Oliva
2004-12-01 19:54                             ` Matthew Wilcox
2004-12-02  3:58                             ` Linus Torvalds
2004-12-02  4:28                               ` Bernd Eckenfels
2004-12-02  5:11                                 ` Peter Williams
2004-12-02 11:30                                   ` Tomas Carnecky
2004-12-02 22:17                                     ` Peter Williams
2004-12-02  9:29                               ` Pekka Enberg
2004-12-02 15:35                                 ` Linus Torvalds
2004-12-02 21:46                                   ` Pekka Enberg
2004-11-30 15:35               ` David Woodhouse
2004-11-30 15:58                 ` Linus Torvalds
2004-11-30 16:28                   ` David Woodhouse
2004-11-30 16:53                     ` Linus Torvalds
2004-11-30 17:52                       ` David Woodhouse
2004-11-30 18:21                         ` Linus Torvalds
2004-11-30 20:28                           ` Mariusz Mazur
2004-11-30 20:47                             ` Linus Torvalds
2004-11-30 22:33                               ` Sam Ravnborg
2004-11-30 22:44                                 ` Mariusz Mazur
2004-11-30 23:03                                   ` Al Viro
2004-11-30 23:08                                     ` Mariusz Mazur
2004-12-01  5:23                                       ` Sam Ravnborg
2004-12-01 10:52                                         ` Mariusz Mazur
2004-12-01 18:08                                           ` Sam Ravnborg
2004-11-30 23:13                                     ` Alexandre Oliva
2004-11-30 23:36                                     ` Chris Friesen
2004-12-01  5:50                               ` H. Peter Anvin
2004-11-30 22:48                           ` Matt Mackall
2004-11-30 22:51                             ` Christoph Hellwig
2004-11-30 23:09                               ` Chris Friesen
2004-11-30 23:35                                 ` Matt Mackall
2004-12-01  0:23                                 ` Miquel van Smoorenburg
2004-11-30 22:55                             ` Linus Torvalds
2004-11-30 23:29                               ` Matt Mackall
2004-12-03  1:03                                 ` Krzysztof Halasa
2004-11-30 23:51                               ` David Woodhouse
2004-11-30 17:55                       ` Al Viro
2004-11-30 16:33                 ` David Howells
2004-11-30 16:57                   ` Linus Torvalds
2004-11-30 15:33             ` David Howells
2004-11-30 15:51               ` Linus Torvalds
2004-11-30 20:47                 ` Alexandre Oliva
2004-11-30 20:56                   ` Linus Torvalds
2004-11-30 21:30                     ` Alexandre Oliva
2004-11-30 21:39                       ` Alexandre Oliva
2004-11-30 23:33                         ` Matthew Wilcox
2004-11-30 22:25                       ` Linus Torvalds
2004-11-30 22:34                         ` David Woodhouse
2004-11-30 22:51                           ` Linus Torvalds
2004-11-30 23:50                             ` David Woodhouse
2004-12-01  0:10                               ` Linus Torvalds
2004-12-01  0:18                                 ` Linus Torvalds
2004-12-01  8:10                                   ` Arjan van de Ven
2004-12-01  0:24                                 ` David Woodhouse
2004-12-01  0:37                                   ` Linus Torvalds
2004-12-01  0:47                                     ` David Woodhouse
2004-12-01  0:57                                       ` Linus Torvalds
2004-12-01  1:06                                         ` David Woodhouse
2004-12-01  1:23                                           ` Linus Torvalds
2004-12-01  6:48                                           ` BAIN
2004-12-01  2:02                               ` cdrom.h (was Re: [RFC] Splitting kernel headers...) Matthew Wilcox
2004-11-30 22:52                         ` [RFC] Splitting kernel headers and deprecating __KERNEL__ Mariusz Mazur
2004-11-30 23:05                           ` Linus Torvalds
2004-11-30 23:05                         ` Alexandre Oliva
2004-11-30 16:18               ` David Howells
2004-12-14  5:51           ` Werner Almesberger
2004-12-14 15:49             ` Linus Torvalds
2004-12-14 16:50               ` Werner Almesberger
2004-12-14 17:58                 ` Linus Torvalds
2004-12-14 21:46                   ` Werner Almesberger
2004-12-14 21:58                     ` Linus Torvalds
2004-12-14 23:49                     ` Krzysztof Halasa
2004-12-15  0:09                       ` Werner Almesberger
2004-12-16  0:58                         ` Krzysztof Halasa
2004-12-14 19:23                 ` Horst von Brand
2004-12-14 19:45                   ` Sam Ravnborg
2004-12-14 19:58                     ` Linus Torvalds
2004-12-14 20:25                       ` Andreas Schwab
2004-12-14 20:33                         ` Linus Torvalds
2004-12-14 16:55               ` Andreas Schwab
2004-12-14 17:59                 ` Linus Torvalds
2004-11-27 20:39   ` Sam Ravnborg
2004-11-25 18:20 ` Alexandre Oliva
2004-11-25 21:01   ` Matthew Wilcox
2004-11-25 22:35     ` David Woodhouse
2004-11-26 11:42       ` Alexandre Oliva
2004-11-27  0:13       ` Andreas Steinmetz
2004-11-27  0:26         ` David Woodhouse
2004-11-27  0:30           ` Andreas Steinmetz
2004-11-27  0:38             ` David Woodhouse
2004-11-26 11:47     ` Alexandre Oliva
2004-11-26 11:53       ` David Woodhouse
2004-11-27 16:12       ` Krzysztof Halasa
2004-11-26 11:58   ` David Howells
2004-11-26 12:00     ` David Woodhouse
2004-11-26 14:19       ` Matthew Wilcox
2004-11-26 14:33         ` David Woodhouse
2004-11-27 12:10           ` Alexander Stohr
2004-11-27  3:46         ` Adam Heath
2004-11-30 12:18         ` Horst von Brand
2004-11-29 16:34     ` Alexandre Oliva
2004-11-25 18:45 ` Matthew Wilcox
2004-11-26 11:52 ` David Howells
2004-11-26 23:56 ` Grzegorz Kulewski
2004-11-27  1:27   ` Tomas Carnecky
2004-11-27  1:41     ` Grzegorz Kulewski
2004-11-27  2:03       ` Tomas Carnecky
2004-11-27  3:07 ` Kyle Moffett
2004-11-27  4:29 ` Tonnerre
2004-11-27  3:51   ` Matthew Wilcox
2004-11-27 16:47     ` Randy.Dunlap
2004-11-27 17:16       ` David Woodhouse
2004-11-27 22:53         ` Arnd Bergmann
2004-11-27 23:12           ` David Woodhouse
2004-11-27 23:27           ` Randy.Dunlap
2004-11-27 23:32             ` David Woodhouse
2004-11-27 23:56               ` Arnd Bergmann
2004-11-28  7:13           ` Arjan van de Ven
2004-11-28 12:03             ` Arnd Bergmann
2004-11-28 12:19               ` Arjan van de Ven
2004-11-28 16:34                 ` Kevin Puetz
2004-11-28 12:28               ` Wichert Akkerman
2004-11-28 12:44                 ` Arjan van de Ven
2004-11-28 16:21             ` Jakub Jelinek
2004-11-28  7:20           ` Christoph Hellwig
2004-11-27  4:05   ` Chris Friesen
2004-11-27  4:30   ` Tomas Carnecky
2004-11-27 21:03 ` Sam Ravnborg
2004-11-27 21:11   ` Andreas Steinmetz
2004-11-27 21:19     ` Sam Ravnborg
2004-11-27 21:49       ` Andreas Steinmetz
2004-11-27 23:21         ` Arnd Bergmann
2004-11-28 13:24           ` Andreas Steinmetz
2004-11-28 13:40             ` Arnd Bergmann
2004-11-29 11:08 ` David Howells
2004-11-29 20:01 ` H. Peter Anvin
2004-11-30  0:34   ` Kyle Moffett
2004-11-30  0:51     ` H. Peter Anvin
2004-12-14  7:07   ` Werner Almesberger
2004-11-27 19:43 Dan Kegel
2004-11-27 21:06 ` Sam Ravnborg
     [not found] <34Xo6-2P0-19@gated-at.bofh.it>
     [not found] ` <35i9f-2vZ-25@gated-at.bofh.it>
     [not found]   ` <35iLS-2Uo-1@gated-at.bofh.it>
     [not found]     ` <35kb6-46Q-25@gated-at.bofh.it>
2004-11-28  0:55       ` ak
2004-11-28 13:32         ` Arnd Bergmann
2004-11-30  1:43 Jean Tourrilhes
2004-11-30  4:22 ` Linus Torvalds
2004-11-30  6:51   ` H. Peter Anvin

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git
	git clone --mirror https://lore.kernel.org/lkml/9 lkml/git/9.git
	git clone --mirror https://lore.kernel.org/lkml/10 lkml/git/10.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git