RE: [Acpi] Re: ACPI fundamental locking problems

RE: [Acpi] Re: ACPI fundamental locking problems

[Dave J Woolley]

> From:	Alan Cox [SMTP:alan@lxorguk.ukuu.org.uk]
> 
> The goal isnt a technical nit, its to avoid loading 300Kbytes of crud
> (which 
> should mostly be in user space anyway) on the 99.9% of machines where we
> dont
> need it.
[DJW:]  
I argued this at the very beginning, but there was a very strong
view that you needed to run most of the code before you had a user
space to run it in.  I've not followed things closely enough to 
know whether or not this is really true and whether or not it is
inevitable, or just a flaw in the ACPI design.

My feeling has been that ACPI has violated the minimum privilege
concept from the beginning, although I think putting stuff in drivers
that could be at user level is not htat uncommon in Linux.

-- 
--------------------------- DISCLAIMER ---------------------------------
Any views expressed in this message are those of the individual sender,
except where the sender specifically states them to be the views of BTS.

>  

Re: [Acpi] Re: ACPI fundamental locking problems

[Alan Cox]

> I argued this at the very beginning, but there was a very strong
> view that you needed to run most of the code before you had a user
> space to run it in.  I've not followed things closely enough to 

That bit is clearly untrue.

> My feeling has been that ACPI has violated the minimum privilege
> concept from the beginning, although I think putting stuff in drivers
> that could be at user level is not htat uncommon in Linux.

It still seems to lack even basic checks that the writes AML does are into
E820 NV or device space only - also while fixing it the ACPI guys might note
that a write close to 0xFFFFFFFF will lead to a failed ioremap and a nasty
mess because they ioremap a fixed sized window that will wrap

Re: [Acpi] Re: ACPI fundamental locking problems

[Linus Torvalds]

On Wed, 4 Jul 2001, Alan Cox wrote:
>
> > I argued this at the very beginning, but there was a very strong
> > view that you needed to run most of the code before you had a user
> > space to run it in.  I've not followed things closely enough to
>
> That bit is clearly untrue.

It's untrue only in the sense of "it is technically possible to do it",
but it _is_ true that right now it's not very convenient to set up an easy
initrd system.

We migth want to just make initrd a built-in thing in the kernel,
something that you simply cannot avoid. A lot of these things (ie dhcp for
NFS root etc) are right now done in kernel space, simply because we don't
want to depend on initrd, and people want to use old loaders.

I don't like the current initrd very much myself, I have to admit. I'm not
going to accept a "you have to have a ramdisk" approach - I think the
ramdisks are really broken.

But I've seen a "populate ramfs from a tar-file built into 'bzImage'"
patch somewhere, and that would be a whole lot more palatable to me.

If anybody were to send me a patch that just unconditionally does this, I
would probably not be adverse to putting it into 2.5.x. We have all the
infrastructure to make all this a lot cleaner than it used to be (ie the
"pivot_root()" stuff etc means that we can _truly_ do things from user
mode, with no magic kernel flags).

But if we do this, then we should _truly_ get rid of all the root device
etc setup crap (and the "search for init" etc stuff - it _is_ going to be
there, and THAT process is the one that should then search for the real
init once it has booted).

That, together with reasonable interfaces to let ACPI set irq data for the
kernel etc, might make moving ACPI back into user space possible in
_practice_ and not just in theory.

		Linus

Re: [Acpi] Re: ACPI fundamental locking problems

[Helge Hafting]

Linus Torvalds wrote:
[...]
> We migth want to just make initrd a built-in thing in the kernel,
> something that you simply cannot avoid. A lot of these things (ie dhcp for
> NFS root etc) are right now done in kernel space, simply because we don't
> want to depend on initrd, and people want to use old loaders.
> 
> I don't like the current initrd very much myself, I have to admit. I'm not
> going to accept a "you have to have a ramdisk" approach - I think the
> ramdisks are really broken.
> 
> But I've seen a "populate ramfs from a tar-file built into 'bzImage'"
> patch somewhere, and that would be a whole lot more palatable to me.
> 
> If anybody were to send me a patch that just unconditionally does this, I
> would probably not be adverse to putting it into 2.5.x. We have all the
> infrastructure to make all this a lot cleaner than it used to be (ie the
> "pivot_root()" stuff etc means that we can _truly_ do things from user
> mode, with no magic kernel flags).
> 
I am fine with "You have to use initrd (or similiar) _if_ you want this
feature."
But please don't make initrd mandatory for those of us who don't
need ACPI, don't need dhcp before mounting disks and so on.

I hope the "fs-less" kernel image still will be possible for those
of us who have a simple setup.

Helge Hafting

Re: [Acpi] Re: ACPI fundamental locking problems

[Jeff Garzik]

Helge Hafting wrote:
> I am fine with "You have to use initrd (or similiar) _if_ you want this
> feature."
> But please don't make initrd mandatory for those of us who don't
> need ACPI, don't need dhcp before mounting disks and so on.

I've always thought it would be neat to do:

	cat bzImage initrd.tar.gz > vmlinuz
	rdev --i-have-a-tarball-piggyback vmlinuz

Linking into the image is easy for hackers, but why not make it
scriptable and super-easy for end users?  x86 already has the rdev
utility to mark a kernel image as having certain flags.  It could even
be a command line option, "inittgz" or somesuch, telling us that a
gzip-format tarball immediately follows the end of our ELF image.

I wonder if any bootloader mods would be needed at all to do this... 
AFAICS you just need to make sure the kernel doesn't trample the
piggyback'd data.

-- 
Jeff Garzik      | Thalidomide, eh? 
Building 1024    | So you're saying the eggplant has an accomplice?
MandrakeSoft     |

Re: [Acpi] Re: ACPI fundamental locking problems

[Alan Cox]

> But please don't make initrd mandatory for those of us who don't
> need ACPI, don't need dhcp before mounting disks and so on.
> 
> I hope the "fs-less" kernel image still will be possible for those
> of us who have a simple setup.

If we can do that kind of early boot user space then stuff like finding the
root file system and possibly even the initial pnpbios scanners belong in
user space. So you would want it for all boxes.

Equally you would want it to be completely trivial - it has to be a case of
a default make bzImage popping out a completely perfect base initrd and
make bzLilo adding both

Re: [Acpi] Re: ACPI fundamental locking problems

[Alexander Viro]

On Thu, 5 Jul 2001, Helge Hafting wrote:

> Linus Torvalds wrote:
> [...]
> > We migth want to just make initrd a built-in thing in the kernel,
> > something that you simply cannot avoid. A lot of these things (ie dhcp for
> > NFS root etc) are right now done in kernel space, simply because we don't
> > want to depend on initrd, and people want to use old loaders.
> > 
> > I don't like the current initrd very much myself, I have to admit. I'm not
> > going to accept a "you have to have a ramdisk" approach - I think the
> > ramdisks are really broken.
> > 
> > But I've seen a "populate ramfs from a tar-file built into 'bzImage'"
> > patch somewhere, and that would be a whole lot more palatable to me.
> > 
> > If anybody were to send me a patch that just unconditionally does this, I
> > would probably not be adverse to putting it into 2.5.x. We have all the
> > infrastructure to make all this a lot cleaner than it used to be (ie the
> > "pivot_root()" stuff etc means that we can _truly_ do things from user
> > mode, with no magic kernel flags).

Open 2.5 and I'm starting to feed that stuff in pieces...

> I am fine with "You have to use initrd (or similiar) _if_ you want this
> feature."

"Similar" == ramfs.

> But please don't make initrd mandatory for those of us who don't
> need ACPI, don't need dhcp before mounting disks and so on.

How about "don't want to keep special-case code for mounting root in your
kernel"? It's more than ramfs, BTW, and rm(1) on ramfs frees memory just
fine.

Re: [Acpi] Re: ACPI fundamental locking problems

[Linus Torvalds]

On Thu, 5 Jul 2001, Helge Hafting wrote:
>
> I am fine with "You have to use initrd (or similiar) _if_ you want this
> feature."

Nope.

I do not want to maintain two interfaces. If we make user space the way to
do these things, then we will do pretty much most of the driver setup etc
in user space. We'd have to: we'd enter user space before drivers have had
a chance to initialize, exactly because "features like these" can change
the device mappings etc.

And I don't want to have two completely different bootup paths.

> But please don't make initrd mandatory for those of us who don't
> need ACPI, don't need dhcp before mounting disks and so on.

You would never even know the difference. You'd do a "make bzImage", and
the default filesystem would just be embedded into the image. By default
it probably doesn't need to do much - although things like the BIOS DPMI
scan etc would surely be good to get rid of.

Why complain about that?

			Linus

Re: [Acpi] Re: ACPI fundamental locking problems

[Benjamin Herrenschmidt]

>
>Nope.
>
>I do not want to maintain two interfaces. If we make user space the way to
>do these things, then we will do pretty much most of the driver setup etc
>in user space. We'd have to: we'd enter user space before drivers have had
>a chance to initialize, exactly because "features like these" can change
>the device mappings etc.
>
>And I don't want to have two completely different bootup paths.

I agree. Also, having this userland step would help for things like
booting from an FireWire or USB hard disk. I hacked the SBP2 (FW)
driver to be useable as a boot device, but this involved adding an
ugly schedule() loop for a couple of seconds before mouting root
in order to leave some time for the drive to be probed. Also, on
such dynamic busses, you can't really know which device major/minor
a given drive will be assigned.

Having a userland mecanism here would allow waiting for all devices
to be probed, reading of the disk GUID (on fw at least) to figure
out where is the real root device, etc... Even displaying a nice
UI to let the user pick a root device is none is found, etc...

So your idea fixes more than just the ACPI problems ;)

Ben.

Re: [Acpi] Re: ACPI fundamental locking problems

[Helge Hafting]

Linus Torvalds wrote:
 
> You would never even know the difference. You'd do a "make bzImage", and
> the default filesystem would just be embedded into the image. By default
> it probably doesn't need to do much - although things like the BIOS DPMI
> scan etc would surely be good to get rid of.
> 
> Why complain about that?

I am convinced.  I misunderstood, thinking there was a big change just
for
ACPI which I and many others don't use.  Thanks for clearing things up.

Helge Hafting

Re: [Acpi] Re: ACPI fundamental locking problems

[Alan Cox]

> I am convinced.  I misunderstood, thinking there was a big change just
> for
> ACPI which I and many others don't use.  Thanks for clearing things up.

It solves a few long standing arguments too - we can slap .config in it 
ending the long standing /proc/config argument without using any ram except
when people care

Re: [Acpi] Re: ACPI fundamental locking problems

[Daniel Phillips]

On Friday 06 July 2001 13:16, Alan Cox wrote:
> > I am convinced.  I misunderstood, thinking there was a big change just
> > for
> > ACPI which I and many others don't use.  Thanks for clearing things up.
>
> It solves a few long standing arguments too - we can slap .config in it
> ending the long standing /proc/config argument without using any ram except
> when people care

The same for System.map

--
Daniel

Re: [Acpi] Re: ACPI fundamental locking problems

[Pavel Machek]

Hi!

> > But please don't make initrd mandatory for those of us who don't
> > need ACPI, don't need dhcp before mounting disks and so on.
> 
> You would never even know the difference. You'd do a "make bzImage", and
> the default filesystem would just be embedded into the image. By default
> it probably doesn't need to do much - although things like the BIOS DPMI
> scan etc would surely be good to get rid of.
> 
> Why complain about that?

I'm afraid it will make bzImage way bigger. And making bzImage build
dependend on libc is not nice... Or are you going with "userspace but
no libc" way? That would solve size, but such userspace
is... well.. not too different from kernel space ;-).
								Pavel
-- 
I'm pavel@ucw.cz. "In my country we have almost anarchy and I don't care."
Panos Katsaloulis describing me w.r.t. patents at discuss@linmodems.org

Re: [Acpi] Re: ACPI fundamental locking problems

[Eugene Crosser]

In article <Pine.LNX.4.33.0107040956310.1668-100000@penguin.transmeta.com>,
        Linus Torvalds <torvalds@transmeta.com> writes:

> I don't like the current initrd very much myself, I have to admit. I'm not
> going to accept a "you have to have a ramdisk" approach - I think the
> ramdisks are really broken.
> 
> But I've seen a "populate ramfs from a tar-file built into 'bzImage'"
> patch somewhere, and that would be a whole lot more palatable to me.

Doesn't the approach "treat a chunk of data built into bzImage as
populated ramfs" look cleaner?  No need to fiddle with tar format,
no copying data from place to place.

Eugene

Re: [Acpi] Re: ACPI fundamental locking problems

[Alexander Viro]

On 7 Jul 2001, Eugene Crosser wrote:

> Doesn't the approach "treat a chunk of data built into bzImage as
> populated ramfs" look cleaner?  No need to fiddle with tar format,
> no copying data from place to place.

What the hell _is_ "populated ramfs"? The thing doesn't live in array
of blocks. Its directory structure consists of a bunch of dentries.
Permissions/ownership/timestamps are in a bunch of struct inode -
sitting in icache and allocated in normal way. Regular files are
entirely in pagecache, ditto for symlinks.

Ramfs has no backing store. At all. That's precisely what remains of
filesystem if you take backing store away - everything is in VFS/VM caches.

Re: [Acpi] Re: ACPI fundamental locking problems

[Eugene Crosser]

In article <Pine.GSO.4.21.0107070727030.24836-100000@weyl.math.psu.edu>,
        Alexander Viro <viro@math.psu.edu> writes:

>> Doesn't the approach "treat a chunk of data built into bzImage as
>> populated ramfs" look cleaner?  No need to fiddle with tar format,
>> no copying data from place to place.
> 
> What the hell _is_ "populated ramfs"? The thing doesn't live in array
> of blocks. Its directory structure consists of a bunch of dentries.

I am stupid.  But the point still stays: having an image of pre-populated
filesystem (some other than ramfs) that you only need to load into
RAM seems more sutable than parsing tar format.  Maybe (probably) I am
missing something.

Eugene

Re: [Acpi] Re: ACPI fundamental locking problems

[Jeff Garzik]

Eugene Crosser wrote:
> 
> In article <Pine.GSO.4.21.0107070727030.24836-100000@weyl.math.psu.edu>,
>         Alexander Viro <viro@math.psu.edu> writes:
> 
> >> Doesn't the approach "treat a chunk of data built into bzImage as
> >> populated ramfs" look cleaner?  No need to fiddle with tar format,
> >> no copying data from place to place.
> >
> > What the hell _is_ "populated ramfs"? The thing doesn't live in array
> > of blocks. Its directory structure consists of a bunch of dentries.
> 
> I am stupid.  But the point still stays: having an image of pre-populated
> filesystem (some other than ramfs) that you only need to load into
> RAM seems more sutable than parsing tar format.  Maybe (probably) I am
> missing something.

Yeah -- we build all this stuff dynamically.  struct file, struct inode,
etc.  You could store them on disk as they would be represented in
memory, but this would be incredibly inefficient because of all the
runtime structures unnecessary on disk, and because of all the fixups
and checks you would have to perform on the data in the images after
they magically appear in memory.

Reading a tarball is the distillation of what you describe into
efficient form :)

-- 
Jeff Garzik      | A recent study has shown that too much soup
Building 1024    | can cause malaise in laboratory mice.
MandrakeSoft     |

Re: [Acpi] Re: ACPI fundamental locking problems

[Daniel Phillips]

On Saturday 07 July 2001 15:50, Jeff Garzik wrote:
> Eugene Crosser wrote:
> > In article
> > <Pine.GSO.4.21.0107070727030.24836-100000@weyl.math.psu.edu>,
> >
> >         Alexander Viro <viro@math.psu.edu> writes:
> > >> Doesn't the approach "treat a chunk of data built into bzImage
> > >> as populated ramfs" look cleaner?  No need to fiddle with tar
> > >> format, no copying data from place to place.
> > >
> > > What the hell _is_ "populated ramfs"? The thing doesn't live in
> > > array of blocks. Its directory structure consists of a bunch of
> > > dentries.
> >
> > I am stupid.  But the point still stays: having an image of
> > pre-populated filesystem (some other than ramfs) that you only need
> > to load into RAM seems more sutable than parsing tar format.  Maybe
> > (probably) I am missing something.
>
> Yeah -- we build all this stuff dynamically.  struct file, struct
> inode, etc.  You could store them on disk as they would be
> represented in memory, but this would be incredibly inefficient
> because of all the runtime structures unnecessary on disk, and
> because of all the fixups and checks you would have to perform on the
> data in the images after they magically appear in memory.

Not to mention internal fragmentation.

> Reading a tarball is the distillation of what you describe into
> efficient form :)

/me downloads tar file definition

Um, gnu tar or posix tar? or some new, improved tar?

--
Daniel

Re: [Acpi] Re: ACPI fundamental locking problems

[Jamie Lokier]

Daniel Phillips wrote:
> > Reading a tarball is the distillation of what you describe into
> > efficient form :)
> 
> /me downloads tar file definition
> 
> Um, gnu tar or posix tar? or some new, improved tar?

I suggest cpio, which is more compact and in some ways more standard.
(tar has a silly pad-to-multiple-of-512-byte per file rule, which is
inappropriate for this).  GNU cpio creates cpio format just fine.

-- Jamie

Re: [Acpi] Re: ACPI fundamental locking problems

[Alexander Viro]

On Sat, 7 Jul 2001, Jamie Lokier wrote:

> Daniel Phillips wrote:
> > > Reading a tarball is the distillation of what you describe into
> > > efficient form :)
> > 
> > /me downloads tar file definition
> > 
> > Um, gnu tar or posix tar? or some new, improved tar?
> 
> I suggest cpio, which is more compact and in some ways more standard.
> (tar has a silly pad-to-multiple-of-512-byte per file rule, which is
> inappropriate for this).  GNU cpio creates cpio format just fine.

GNU cpio is a race-ridden unmaintained pile of junk. Look at the size
of, say it, Debian patch to upstream source. Then try to read the
patched code.  Quite a few of us simply don't have that FPOS on their
boxen.

Using cpio archive layout is OK, but _please_, don't make it dependent
on GNU cpio.

Re: [Acpi] Re: ACPI fundamental locking problems

[Jonathan Lundell]

At 3:26 AM -0400 2001-07-08, Alexander Viro wrote:
>On Sat, 7 Jul 2001, Jamie Lokier wrote:
>
>>  Daniel Phillips wrote:
>>  > > Reading a tarball is the distillation of what you describe into
>>  > > efficient form :)
>>  >
>>  > /me downloads tar file definition
>>  >
>>  > Um, gnu tar or posix tar? or some new, improved tar?
>>
>>  I suggest cpio, which is more compact and in some ways more standard.
>>  (tar has a silly pad-to-multiple-of-512-byte per file rule, which is
>>  inappropriate for this).  GNU cpio creates cpio format just fine.
>
>GNU cpio is a race-ridden unmaintained pile of junk. Look at the size
>of, say it, Debian patch to upstream source. Then try to read the
>patched code.  Quite a few of us simply don't have that FPOS on their
>boxen.
>
>Using cpio archive layout is OK, but _please_, don't make it dependent
>on GNU cpio.

If size is an issue (and of course it is), presumably the archive 
would be compressed. As long as tar can be convinced to pad with 
(say) nulls, the padding shouldn't have that much of an impact on 
archive size.
-- 
/Jonathan Lundell.

Re: [Acpi] Re: ACPI fundamental locking problems

[Anthony DeBoer]

Jamie Lokier <lk@tantalophile.demon.co.uk> wrote:
>(tar has a silly pad-to-multiple-of-512-byte per file rule, which is
>inappropriate for this).  GNU cpio creates cpio format just fine.

Tarballs are almost universally compressed, and that pad squishes fairly
well then.  Certainly in kernel-piggyback mode that step wouldn't get
omitted.

-- 
Anthony de Boer, curator, Anthony's Home for Aged Computing Machinery
<adb@leftmind.net>

Re: [Acpi] Re: ACPI fundamental locking problems

[Steve VanDevender]

Jamie Lokier writes:
 > (tar has a silly pad-to-multiple-of-512-byte per file rule, which is
 > inappropriate for this).

If you remember that 'tar' means "tape archiver", and that at the time
it was written the standard tape block size was 512 bytes, the rule
isn't silly at all, although it may be undesirable overhead for modern
uses of 'tar'.

Re: [Acpi] Re: ACPI fundamental locking problems

[Jeff Garzik]

Eugene Crosser wrote:
> 
> In article <Pine.LNX.4.33.0107040956310.1668-100000@penguin.transmeta.com>,
>         Linus Torvalds <torvalds@transmeta.com> writes:
> 
> > I don't like the current initrd very much myself, I have to admit. I'm not
> > going to accept a "you have to have a ramdisk" approach - I think the
> > ramdisks are really broken.
> >
> > But I've seen a "populate ramfs from a tar-file built into 'bzImage'"
> > patch somewhere, and that would be a whole lot more palatable to me.
> 
> Doesn't the approach "treat a chunk of data built into bzImage as
> populated ramfs" look cleaner?  No need to fiddle with tar format,
> no copying data from place to place.

So tell me, how do you populate ramfs without a format which tells you
what path and which permissions to assign each file?  That's exactly
what tar is.

-- 
Jeff Garzik      | A recent study has shown that too much soup
Building 1024    | can cause malaise in laboratory mice.
MandrakeSoft     |

Re: [Acpi] Re: ACPI fundamental locking problems

[Mike Touloumtzis]

On Sat, Jul 07, 2001 at 07:34:38AM -0400, Jeff Garzik wrote:
> Eugene Crosser wrote:
> > 
> > Doesn't the approach "treat a chunk of data built into bzImage as
> > populated ramfs" look cleaner?  No need to fiddle with tar format,
> > no copying data from place to place.
> 
> So tell me, how do you populate ramfs without a format which tells you
> what path and which permissions to assign each file?  That's exactly
> what tar is.

Would it be possible to use a cramfs image in vmlinux (i.e. real
filesystem image, not an in-kernel-structures fs like ramfs), and map
it directly from the kernel image (it would have to be suitably aligned,
of course)?

This would allow demand paging of files in the image (not too important
for a minimal boot fs, admittedly), and would allow text pages to be
dropped under VM pressure (nice for a fs which holds substantial amounts
of boot-time-only code).

miket

Re: [Acpi] Re: ACPI fundamental locking problems

[Jamie Lokier]

Mike Touloumtzis wrote:
> > > Doesn't the approach "treat a chunk of data built into bzImage as
> > > populated ramfs" look cleaner?  No need to fiddle with tar format,
> > > no copying data from place to place.
> > 
> > So tell me, how do you populate ramfs without a format which tells you
> > what path and which permissions to assign each file?  That's exactly
> > what tar is.
> 
> Would it be possible to use a cramfs image in vmlinux (i.e. real
> filesystem image, not an in-kernel-structures fs like ramfs), and map
> it directly from the kernel image (it would have to be suitably aligned,
> of course)?
> 
> This would allow demand paging of files in the image (not too important
> for a minimal boot fs, admittedly), and would allow text pages to be
> dropped under VM pressure (nice for a fs which holds substantial amounts
> of boot-time-only code).

Yes that would work, and it would work on machines with less RAM too.
You would want to remove the cramfs filesystem code when you're done though.

-- Jamie

Re: [Acpi] Re: ACPI fundamental locking problems

[arjan]

In article <20010707235329.A10256@pcep-jamie.cern.ch> you wrote:
>> Would it be possible to use a cramfs image in vmlinux (i.e. real
>> filesystem image, not an in-kernel-structures fs like ramfs), and map
>> it directly from the kernel image (it would have to be suitably aligned,
>> of course)?

> Yes that would work, and it would work on machines with less RAM too.
> You would want to remove the cramfs filesystem code when you're done though.

Until you pxe-boot your kernel over the network........

Re: [Acpi] Re: ACPI fundamental locking problems

[Jamie Lokier]

arjan@fenrus.demon.nl wrote:
> >> Would it be possible to use a cramfs image in vmlinux (i.e. real
> >> filesystem image, not an in-kernel-structures fs like ramfs), and map
> >> it directly from the kernel image (it would have to be suitably aligned,
> >> of course)?
> 
> > Yes that would work, and it would work on machines with less RAM too.
> > You would want to remove the cramfs filesystem code when you're done though.
> 
> Until you pxe-boot your kernel over the network........

I don't see a problem with that.  pxe-boot loads vmlinuz, kernel boots
and finds cramfs image inside itself.  Problem?

-- Jamie

Re: [Acpi] Re: ACPI fundamental locking problems

[Mike Touloumtzis]

On Sat, Jul 07, 2001 at 11:53:29PM +0200, Jamie Lokier wrote:
> Mike Touloumtzis wrote:
> > 
> > Would it be possible to use a cramfs image in vmlinux (i.e. real
> > filesystem image, not an in-kernel-structures fs like ramfs), and map
> > it directly from the kernel image (it would have to be suitably aligned,
> > of course)?
> > 
> > This would allow demand paging of files in the image (not too important
> > for a minimal boot fs, admittedly), and would allow text pages to be
> > dropped under VM pressure (nice for a fs which holds substantial amounts
> > of boot-time-only code).
> 
> Yes that would work, and it would work on machines with less RAM too.
> You would want to remove the cramfs filesystem code when you're done though.

Some of the files in the boot time FS would need to
be kept around, such as the ACPI code, right?

miket

Re: [Acpi] Re: ACPI fundamental locking problems

[Jamie Lokier]

Mike Touloumtzis wrote:
> > Yes that would work, and it would work on machines with less RAM too.
> > You would want to remove the cramfs filesystem code when you're done though.
> 
> Some of the files in the boot time FS would need to
> be kept around, such as the ACPI code, right?

Perhaps.  They could be copied to a ramfs by the user-space code :-)

On this theme, it's just occured to me that the module loader could be
taught to map ramfs pages directly to module code/data space.  That
would save a little memory.

-- Jamie

Re: [Acpi] Re: ACPI fundamental locking problems

[Keith Owens]

On Sun, 8 Jul 2001 00:14:24 +0200, 
Jamie Lokier <lk@tantalophile.demon.co.uk> wrote:
>On this theme, it's just occured to me that the module loader could be
>taught to map ramfs pages directly to module code/data space.  That
>would save a little memory.

I doubt it.  insmod relocates the code and data sections, discards
sections, inserts a struct module, hooks the module into the existing
change and generally mangles the object before it can be used by the
kernel.  Any change to a page prevents it being mapped against cramfs.
If you had a complete page that was identical before and after insmod
had done its work I would be astonished.

Re: [Acpi] Re: ACPI fundamental locking problems

[Eric W. Biederman]

Linus Torvalds <torvalds@transmeta.com> writes:

> On Wed, 4 Jul 2001, Alan Cox wrote:

> We migth want to just make initrd a built-in thing in the kernel,
> something that you simply cannot avoid. A lot of these things (ie dhcp for
> NFS root etc) are right now done in kernel space, simply because we don't
> want to depend on initrd, and people want to use old loaders.

That and the linux tools for making small binaries are relatively
immature.

> I don't like the current initrd very much myself, I have to admit. I'm not
> going to accept a "you have to have a ramdisk" approach - I think the
> ramdisks are really broken.
> 
> But I've seen a "populate ramfs from a tar-file built into 'bzImage'"
> patch somewhere, and that would be a whole lot more palatable to me.

To some extent I'd prefer to build the tar-file into vmlinux as that
makes it a multi architecture solution.  I don't like the fact that
rdev only works on x86.
 
> If anybody were to send me a patch that just unconditionally does this, I
> would probably not be adverse to putting it into 2.5.x. We have all the
> infrastructure to make all this a lot cleaner than it used to be (ie the
> "pivot_root()" stuff etc means that we can _truly_ do things from user
> mode, with no magic kernel flags).
>
> But if we do this, then we should _truly_ get rid of all the root device
> etc setup crap (and the "search for init" etc stuff - it _is_ going to be
> there, and THAT process is the one that should then search for the real
> init once it has booted).

A list of issues I can see with doing this right now.

- umounting the initial fs after you have called pivot_root is
  tricky, can we run a program from an internal mount only?
  (We can remove all of the files on the initial fs with rm -rf /
   assuming we are running on ramfs)

- The version of ``preinit'' cannot use glibc, there is too much
  bloat.  uclibc is o.k. but a little immature.  We can probably use
  the infrastructure we have in linux/unistd.h for doing system calls
  from the kernel to remove any dependieces on other packages.  But
  using kernel headers from user space has been outlawed... 

- In the case of console=tty0 console=ttyS0 /dev/console does not
  output to the same locations as printk.

- We must be architecture netural.  Do this only for x86 is
  unacceptable.

- The _init stuff that allows us to throw code after device
  initialization would need to be disabled to some extent because it
  would now depends on code in user space.

> That, together with reasonable interfaces to let ACPI set irq data for the
> kernel etc, might make moving ACPI back into user space possible in
> _practice_ and not just in theory.

Irq tables.  A corrected system memory map.  Builtin ISA devices.
Long term we need is an interface to feed a pre intialized 
``struct device'' (the renamed struct pci_device) tree into the kernel.

Eric

Re: [Acpi] Re: ACPI fundamental locking problems

[Jeff Garzik]

"Eric W. Biederman" wrote:
> Linus Torvalds <torvalds@transmeta.com> writes:
> > On Wed, 4 Jul 2001, Alan Cox wrote:
> > We migth want to just make initrd a built-in thing in the kernel,
> > something that you simply cannot avoid. A lot of these things (ie dhcp for
> > NFS root etc) are right now done in kernel space, simply because we don't
> > want to depend on initrd, and people want to use old loaders.
> 
> That and the linux tools for making small binaries are relatively
> immature.

I am rapidly discovering that :(

I wouldn't mind, for example, having a linker that is smart enough to
eliminate dead code inside code sections, and can rewrite function
prologues to more optimized forms once it knows the entire scope of said
functions.  (ie. an optimizing linker)


> > I don't like the current initrd very much myself, I have to admit. I'm not
> > going to accept a "you have to have a ramdisk" approach - I think the
> > ramdisks are really broken.
> >
> > But I've seen a "populate ramfs from a tar-file built into 'bzImage'"
> > patch somewhere, and that would be a whole lot more palatable to me.
> 
> To some extent I'd prefer to build the tar-file into vmlinux as that
> makes it a multi architecture solution.  I don't like the fact that
> rdev only works on x86.

IMHO rdev -should- work on other platforms.  There -should- be a way to
set flags inside an already-built kernel image.  That concept is not
inherently x86.

That is why I favor

	cat bzImage ramfs.tar.gz > vmlinuz
	rdev vmlinuz i-have-a-ramfs

It's -much- more flexible than actually building the initramfs into the
kernel.

Admittedly I'm biased but IMHO the only reason why people want to lose
flexibility by avoiding this approach is to avoid coming up with an rdev
concept that works on alpha, sparc, ...  :)


> - The version of ``preinit'' cannot use glibc, there is too much
>   bloat.  uclibc is o.k. but a little immature.  We can probably use
>   the infrastructure we have in linux/unistd.h for doing system calls
>   from the kernel to remove any dependieces on other packages.  But
>   using kernel headers from user space has been outlawed...

There should be no need to use linux/unistd.h infrastructure.  dietlibc
is just as small (smaller than uClibc in the cases I've tried), and has
already done this.  They even have dynamic linking going for x86, arm,
sparc, and maybe others.  http://www.fefe.de/dietlibc/


> - We must be architecture netural.  Do this only for x86 is
>   unacceptable.

obviously


> - The _init stuff that allows us to throw code after device
>   initialization would need to be disabled to some extent because it
>   would now depends on code in user space.

I don't see this at all, can you elaborate?


> Irq tables.  A corrected system memory map.  Builtin ISA devices.
> Long term we need is an interface to feed a pre intialized
> ``struct device'' (the renamed struct pci_device) tree into the kernel.

ie. come up with our own firmware->kernel information passing interface
:)

Let's make it better than ACPI this time, shall we?  :)

-- 
Jeff Garzik      | A recent study has shown that too much soup
Building 1024    | can cause malaise in laboratory mice.
MandrakeSoft     |

Re: [Acpi] Re: ACPI fundamental locking problems

[Eric W. Biederman]

Jeff Garzik <jgarzik@mandrakesoft.com> writes:

> "Eric W. Biederman" wrote:
> > That and the linux tools for making small binaries are relatively
> > immature.
> 
> I am rapidly discovering that :(
> 
> I wouldn't mind, for example, having a linker that is smart enough to
> eliminate dead code inside code sections, and can rewrite function
> prologues to more optimized forms once it knows the entire scope of said
> functions.  (ie. an optimizing linker)

I'll worry about the linker when the embedded libc problem is handled.
uClibc is the most mature when I looked several months ago, and unlike dietlibc
it actually tried not to user kernel headers.
 
> > > I don't like the current initrd very much myself, I have to admit. I'm not
> > > going to accept a "you have to have a ramdisk" approach - I think the
> > > ramdisks are really broken.
> > >
> > > But I've seen a "populate ramfs from a tar-file built into 'bzImage'"
> > > patch somewhere, and that would be a whole lot more palatable to me.
> > 
> > To some extent I'd prefer to build the tar-file into vmlinux as that
> > makes it a multi architecture solution.  I don't like the fact that
> > rdev only works on x86.
> 
> IMHO rdev -should- work on other platforms.  There -should- be a way to
> set flags inside an already-built kernel image.  That concept is not
> inherently x86.
> 
> That is why I favor
> 
> 	cat bzImage ramfs.tar.gz > vmlinuz
> 	rdev vmlinuz i-have-a-ramfs
> 
> It's -much- more flexible than actually building the initramfs into the
> kernel.
> 
> Admittedly I'm biased but IMHO the only reason why people want to lose
> flexibility by avoiding this approach is to avoid coming up with an rdev
> concept that works on alpha, sparc, ...  :)

Which is why I mentioned vmlinux.  All platforms build it at some stage.

> > - The version of ``preinit'' cannot use glibc, there is too much
> >   bloat.  uclibc is o.k. but a little immature.  We can probably use
> >   the infrastructure we have in linux/unistd.h for doing system calls
> >   from the kernel to remove any dependieces on other packages.  But
> >   using kernel headers from user space has been outlawed...
> 
> There should be no need to use linux/unistd.h infrastructure.  dietlibc
> is just as small (smaller than uClibc in the cases I've tried), and has
> already done this.  They even have dynamic linking going for x86, arm,
> sparc, and maybe others.  http://www.fefe.de/dietlibc/

I've played with it.  Dietlibc is horrible on alpha.  The basic problem
is dependecies on kernel headers.  uClibc was at least straight forward
to port.  In my backlog of things to do is get the uClibc port to alpha
contributed.

> > - We must be architecture netural.  Do this only for x86 is
> >   unacceptable.
> 
> obviously

I saw plenty of reference to bzImage and vmlinuz, it is only obvious
in hindsight.  

> > - The _init stuff that allows us to throw code after device
> >   initialization would need to be disabled to some extent because it
> >   would now depends on code in user space.
> 
> I don't see this at all, can you elaborate?

Well if you don't know the devices is there, you can't throw away it's
initialization routine.  This problem already exists to some extent with
hot-plug pci devices.  But if you treat the host pci bridge as a hot-plug
pci device, then everything else is as well..

> > Irq tables.  A corrected system memory map.  Builtin ISA devices.
> > Long term we need is an interface to feed a pre intialized
> > ``struct device'' (the renamed struct pci_device) tree into the kernel.
> 
> ie. come up with our own firmware->kernel information passing interface
> :)
> 
> Let's make it better than ACPI this time, shall we?  :)

It's probably the most important thing on my todo list, (with the
exception of convincing AMD it's not a problem to release a few extra
lines of code)...  Though
since it is most grunt work and detail checking I'm not even going to
look at it until I'm back from vacation :)

Eric

Re: [Acpi] Re: ACPI fundamental locking problems

[Andreas Dilger]

Jeff writes:
> I've always thought it would be neat to do:
> 
>       cat bzImage initrd.tar.gz > vmlinuz
>       rdev --i-have-a-tarball-piggyback vmlinuz
> 
> Linking into the image is easy for hackers, but why not make it
> scriptable and super-easy for end users?  x86 already has the rdev
> utility to mark a kernel image as having certain flags.  It could even
> be a command line option, "inittgz" or somesuch, telling us that a
> gzip-format tarball immediately follows the end of our ELF image.

This would be especially handy for network booting: you only need to send
the one file to the client, and it boots the kernel and loads the initrd
without any extra requests/parameters/configuration needed.

> I wonder if any bootloader mods would be needed at all to do this... 
> AFAICS you just need to make sure the kernel doesn't trample the
> piggyback'd data.

Probably not - the kernel would handle all of it.  It sounds like Linus
and Al are in favour of this, so it will likely be in 2.5.early.  Having
the tar be extracted into ramfs has the added benefit that you don't need
to 'pre-configure' ramdisk size, make dd initrd images, or waste memory
that is representing empty fs space.  Conversely, if the root is ramfs
you also don't need to worry about the ramdisk fs being too small if you
need to create some temprary files there...  It is a win in all cases.

Cheers, Andreas
-- 
Andreas Dilger                               Turbolinux filesystem development
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/