faster boots?

faster boots?

[joeja]

Hello, 
    Is there some way of making the linux kernel boot faster?  

    While I know that many people here probably don't reboot there machines often, I live in CA where my electrictiy is still high and see no reason to keep a machine on that is not in use (i.e. while I sleep or am at work).  
  
    I tested freebsd on an old P133Mhz/32Meg ram and it booted faster with the GENERIC kernel than linux did on a AMD 1200Mhz/512Meg ram, which seemed odd.  Linux on that same P133 box also took longer than FreeBSD to boot.  

    If I have a machine that does not change from day to day hardware wise why when I boot the thing do I need to probe the hardware again and again each time?  Would passing more options on the command line help like all the addresses and IRQ's of known hardware?
    Wouldn't it make sense to store this data on the files system? Certainly if something like grub or lilo can figure out how to access a file on the drive the kernel could check for a 'defaults' file or something to get the default irq's, hardware, interrupts, etc from.  Then the kernel could probe these first and if the probe fails proble elsewhere for the device.

  Or is there another way of speeding up the linux kernel boot process?

Joe

Re: faster boots?

[Alan Cox]

>     Is there some way of making the linux kernel boot faster?  

#1: Start less crap at boot time. Obvious but thats frequently most of
    the issue.

For Red Hat if your hardware set up is constant then rpm -e kudzu will do
no harm and avoid the grovelling through the box looking for new toys.

Longer term swsuspend means you can bang alt-sysrq-Z and suspend to disk
without BIOS support

Re: faster boots?

[Andrew Morton]

joeja@mindspring.com wrote:
> 
> Hello,
>     Is there some way of making the linux kernel boot faster?
> 

Joe, you're speaking to a bunch of people who reboot their
boxes ten times an hour or more.  It's an interesting topic.

- If your BIOS has an option to not call the bios functions
  of add-in cards, try using it.  Every card seems to have
  its own inbuilt ten-second keyboard timeout, and this
  option trumps them all.

- Remove as many of those nice vendor-provided initscripts
  as you can.  It sounds like you're running kudzu or
  equivalent?  `rpm -e' or `chkconfig kudzu off'.

- Modify /etc/init.d/rd.c/xinted so that it starts xinetd
  asynchronously.  For some reason, xinetd has a nice
  five-second sleep before allowing the initscripts to
  continue.  I haven't noticed it do anything useful.

- If you're using SCSI, (aic7xxx), go into the kernel
  config system and reduce its "initial bus reset delay
  in milli-seconds".  I use 1500.

If you want to get really radical you could investigate
Richard Gooch's boot scripts which I believe allow all the
initscripts to be launched in parallel.  Which is a good
thing to do.

	http://www.atnf.csiro.au/people/rgooch/linux/boot-scripts/

Once everything is trimmed up and tightened down, you should
have reasonably good boot times.  There isn't a lot more
to be squeezed out of it.

Last year I developed a bunch of code which was designed to
speed up the boot process.  It works as follows:

1: Boot the machine, start your X server or whatever you
   normally do.  Wait for everything to settle down.

2: Load a kernel module and run a userspace app which dumps
   out a directory of your pagecache and buffercache contents
   to a database file.  It's of the form:

	filename:page,page,page,page and
	device:block,block,block,

   The database file is sorted in ascending block order.

3: Next time you reboot, load a different kernel module and
   run a different userspace app which together walk that
   database and preload the pagecache and buffercache.

The theory was lovely.  And I tried all sorts of stuff.  But
the bottom line benefit was only about 10%.  The whole thing
was constrained by buffercache seek time - filesytem metadata.

Oh well.  The best benefit was in fact from launching all
the initscripts in parallel.  Lots of stuff broke because
of the lack of any sort of dependency system, but it was
appreciably quicker.

I guess the greatest benefit would come from reorganising the
layout of the root filesystem's data and metadata so the
pagecache prepopulation doesn't have to seek all over the place.

-

Re: faster boots?

[Piotr Esden-Tempski]

[-- Attachment #1: Type: text/plain, Size: 2392 bytes --]

Hi,

I think that the speed of the kernel boot is not the main speed problem.
You may want to try minit written by Fefe. You can download it and test.
I have not tried it myself but I heave seen Fefe's laptop on 18c3 and it
booted really impresive fast.

cheers Piotr

On Thu, Apr 04, 2002 at 06:54:07PM -0500, joeja@mindspring.com wrote:
> Hello, 
>     Is there some way of making the linux kernel boot faster?  
> 
>     While I know that many people here probably don't reboot there machines often, I live in CA where my electrictiy is still high and see no reason to keep a machine on that is not in use (i.e. while I sleep or am at work).  
>   
>     I tested freebsd on an old P133Mhz/32Meg ram and it booted faster with the GENERIC kernel than linux did on a AMD 1200Mhz/512Meg ram, which seemed odd.  Linux on that same P133 box also took longer than FreeBSD to boot.  
> 
>     If I have a machine that does not change from day to day hardware wise why when I boot the thing do I need to probe the hardware again and again each time?  Would passing more options on the command line help like all the addresses and IRQ's of known hardware?
>     Wouldn't it make sense to store this data on the files system? Certainly if something like grub or lilo can figure out how to access a file on the drive the kernel could check for a 'defaults' file or something to get the default irq's, hardware, interrupts, etc from.  Then the kernel could probe these first and if the probe fails proble elsewhere for the device.
> 
>   Or is there another way of speeding up the linux kernel boot process?
> 
> Joe
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 
bChat2: http://bchat2.bingo-ev.de                       ___  ___  ___  _  _
bChat: http://bchat.bingo-ev.de                        | _ || _ || __|| |//
ROCK LINUX: www.rocklinux.org                          ||_|||| ||||   |  /
-Born to run kill -9 win                               |  _|||_||||__ |  \
-"Ignorance is bliss." (Matrix)                        ||\\ |_LINUX__||_|\\
GPG Public Key Block: http://www.esden.net/me/esden-key-2002-01-17.asc

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: faster boots?

[Jeremy Jackson]

----- Original Message ----- 
From: "Alan Cox" <alan@lxorguk.ukuu.org.uk>
Sent: Thursday, April 04, 2002 4:21 PM
Subject: Re: faster boots?


> >     Is there some way of making the linux kernel boot faster?  
> 
> #1: Start less crap at boot time. Obvious but thats frequently most of
>     the issue.
> 
> For Red Hat if your hardware set up is constant then rpm -e kudzu will do
> no harm and avoid the grovelling through the box looking for new toys.
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
rotfl, btw.  oh my aching sides.  I may have to quote
that.  reminds me of windows 95 hardware detection

May I suggest chkconfig --level 2345 kudzu off, a less invasive way
to achieve this, without burning bridges.  /sbin/kudzu can still be run
manually then, if the hardware does change and you're lazy (and you
trust kudzu :)

Jeremy

Re: faster boots?

[Ross Vandegrift]

> on a AMD 1200Mhz/512Meg ram, which seemed odd.

Is your motherboard one with a Promise Ultra100 controller integrated?

If so, you can shave that time off by recomiling your kernel with
"Special UDMA Feature" and then disabling the Ultra100 bios.  You should
still be able to get UDMA without loading the annoying BIOS.

Ross Vandegrift
ross@willow.seitz.com

Re: faster boots?

[Bernd Eckenfels]

In article <Springmail.0994.1017964447.0.72656900@webmail.atl.earthlink.net> you wrote:
>  Or is there another way of speeding up the linux kernel boot process?

Have you watched the boot process for long pause? SCSI Init and not found
devices can delay. if you make your own kernel with only the required
hardware (and perhaps put a lot into modules which are only used if needed)
this can help. Actually a faster init also does help.

Greetings
Bernd

Re: faster boots?

[Richard Gooch]

Andrew Morton writes:
> Joe, you're speaking to a bunch of people who reboot their
> boxes ten times an hour or more.  It's an interesting topic.
> 
> If you want to get really radical you could investigate
> Richard Gooch's boot scripts which I believe allow all the
> initscripts to be launched in parallel.  Which is a good
> thing to do.
> 
> 	http://www.atnf.csiro.au/people/rgooch/linux/boot-scripts/

Around 20 seconds or less from when the kernel starts init(8) to when
my XDM splash screen is up, last time I checked.

> Last year I developed a bunch of code which was designed to
> speed up the boot process.  It works as follows:
> 
> 1: Boot the machine, start your X server or whatever you
>    normally do.  Wait for everything to settle down.
> 
> 2: Load a kernel module and run a userspace app which dumps
>    out a directory of your pagecache and buffercache contents
>    to a database file.  It's of the form:
> 
> 	filename:page,page,page,page and
> 	device:block,block,block,
> 
>    The database file is sorted in ascending block order.
> 
> 3: Next time you reboot, load a different kernel module and
>    run a different userspace app which together walk that
>    database and preload the pagecache and buffercache.
> 
> The theory was lovely.  And I tried all sorts of stuff.  But
> the bottom line benefit was only about 10%.  The whole thing
> was constrained by buffercache seek time - filesytem metadata.

The problem is that you're not listing the metadata blocks when
building the database, right? And that's because Linus didn't want
such hackery in the kernel. So instead we got the not-very-useful
readahead system call.

> Oh well.  The best benefit was in fact from launching all
> the initscripts in parallel.  Lots of stuff broke because
> of the lack of any sort of dependency system, but it was
> appreciably quicker.

Of course, my boot scripts do the dependency stuff right (actually,
it's the changes I made to simpleinit(8) that make it possible).

> I guess the greatest benefit would come from reorganising the
> layout of the root filesystem's data and metadata so the
> pagecache prepopulation doesn't have to seek all over the place.

Or being able to preload *everything* in ascending order...

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: faster boots?

[Andrew Morton]

Richard Gooch wrote:
> 
> >       http://www.atnf.csiro.au/people/rgooch/linux/boot-scripts/
> 
> Around 20 seconds or less from when the kernel starts init(8) to when
> my XDM splash screen is up, last time I checked.

eww.  You need a doctored filesystem.

> ...
> > The theory was lovely.  And I tried all sorts of stuff.  But
> > the bottom line benefit was only about 10%.  The whole thing
> > was constrained by buffercache seek time - filesytem metadata.
> 
> The problem is that you're not listing the metadata blocks when
> building the database, right? And that's because Linus didn't want
> such hackery in the kernel. So instead we got the not-very-useful
> readahead system call.

No, everything was listed.  pagecache, buffercache.  This
was pre buffercache-in-pagecache.  I tried lots of stuff,
including intermingling pagecache and buffercache reads
in strict LBA order, buffercache first, no buffercache at
all.  Nothing really helped.  Fact is, all those files are
sprinkled all over the disk and a short seek is pretty much
as expensive as a long one.

The code's at http://www.zip.com.au/~akpm/linux/fboot.tar.gz -
it includes a demonstration of the ancient art of reading files
in a kernel module via insmod's standard input :)

> > Oh well.  The best benefit was in fact from launching all
> > the initscripts in parallel.  Lots of stuff broke because
> > of the lack of any sort of dependency system, but it was
> > appreciably quicker.
> 
> Of course, my boot scripts do the dependency stuff right (actually,
> it's the changes I made to simpleinit(8) that make it possible).

Yes, I've looked.  It's nice stuff.  The dependencies are critial.

> > I guess the greatest benefit would come from reorganising the
> > layout of the root filesystem's data and metadata so the
> > pagecache prepopulation doesn't have to seek all over the place.
> 
> Or being able to preload *everything* in ascending order...

I was preloading everything.  I certainly avoided the thought of
taking a *copy* of everything and placing it elsewhere on disk.
Scary coherency problems there.

One thing I did do a while back was to set up a new root filesystem
on a new disk via `tar cf - | (cd /newplace ; tar xf -)'.  But before
doing this I nobbled ext2's directory placement algorithm so
subdirectories in the new fs go in the same blockgroup as the parent.
This sped boots up quite a bit.  Probably the pagecache preload
code would work better with that setup.

Still.  Joe tells me (offlist) that his machine is taking
ages just to get to the "starting init" stage.

-

Re: faster boots?

[Benjamin LaHaise]

On Thu, Apr 04, 2002 at 06:51:46PM -0800, Andrew Morton wrote:
> Still.  Joe tells me (offlist) that his machine is taking
> ages just to get to the "starting init" stage.

I find that on heavily scsi systems: one machine spins each of 13 disks 
up sequentially.  This makes the initial boot take 3-5 minutes before 
init even gets its foot in the door.  If someone made a patch to spin 
up scsi disks on the first access, I'd gladly give it a test. ;-)

		-ben
-- 
"A man with a bass just walked in,
 and he's putting it down
 on the floor."

Re: faster boots?

[Alan Cox]

> I find that on heavily scsi systems: one machine spins each of 13 disks 
> up sequentially.  This makes the initial boot take 3-5 minutes before 
> init even gets its foot in the door.  If someone made a patch to spin 
> up scsi disks on the first access, I'd gladly give it a test. ;-)

Ditto. Especially if it spun them down again when idle for a while. 

The scsi layer does several things serially it could parallelise. It isnt
just disk spin up its also things like initialising all scsi controllers
in parallel.

Alan

Re: faster boots?

[Richard Gooch]

Andrew Morton writes:
> Richard Gooch wrote:
> > 
> > >       http://www.atnf.csiro.au/people/rgooch/linux/boot-scripts/
> > 
> > Around 20 seconds or less from when the kernel starts init(8) to when
> > my XDM splash screen is up, last time I checked.
> 
> eww.  You need a doctored filesystem.

Sorry. I'm not sure what you mean. Are you saying that if I had a
doctored FS, then it would be a lot faster? If so, how much? What are
your boot times?

> > > The theory was lovely.  And I tried all sorts of stuff.  But
> > > the bottom line benefit was only about 10%.  The whole thing
> > > was constrained by buffercache seek time - filesytem metadata.
> > 
> > The problem is that you're not listing the metadata blocks when
> > building the database, right? And that's because Linus didn't want
> > such hackery in the kernel. So instead we got the not-very-useful
> > readahead system call.
> 
> No, everything was listed.  pagecache, buffercache.  This
> was pre buffercache-in-pagecache.  I tried lots of stuff,
> including intermingling pagecache and buffercache reads
> in strict LBA order, buffercache first, no buffercache at
> all.  Nothing really helped.  Fact is, all those files are
> sprinkled all over the disk and a short seek is pretty much
> as expensive as a long one.

Bugger.

> > Of course, my boot scripts do the dependency stuff right (actually,
> > it's the changes I made to simpleinit(8) that make it possible).
> 
> Yes, I've looked.  It's nice stuff.  The dependencies are critial.

<blush> Thanks.

> One thing I did do a while back was to set up a new root filesystem
> on a new disk via `tar cf - | (cd /newplace ; tar xf -)'.  But
> before doing this I nobbled ext2's directory placement algorithm so
> subdirectories in the new fs go in the same blockgroup as the
> parent.  This sped boots up quite a bit.  Probably the pagecache
> preload code would work better with that setup.

Yeah. I guess this nobbling isn't likely to make it into the kernel. I
should fiddle with tar sometime to create everything in the one
directory and move stuff around to the final destination. That would
get around the ext2fs placement misfeature (in this context, anyway).

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: faster boots?

[Richard Gooch]

Alan Cox writes:
> > I find that on heavily scsi systems: one machine spins each of 13 disks 
> > up sequentially.  This makes the initial boot take 3-5 minutes before 
> > init even gets its foot in the door.  If someone made a patch to spin 
> > up scsi disks on the first access, I'd gladly give it a test. ;-)
> 
> Ditto. Especially if it spun them down again when idle for a while. 

Be careful here. I did this for a while with a Maxtor 80 GB IDE drive,
and after a few months, it started making unpleasant noises when
spinning up (lots of clicking and clacking). I went back to continuous
spinning. I'm not in the mood for replacing my drives every few
months :-(

> The scsi layer does several things serially it could parallelise. It
> isnt just disk spin up its also things like initialising all scsi
> controllers in parallel.

Indeed. However, the business of spin-up should be handled either in
the SCSI BIOS extension, or better yet, using auto-start delay. If the
kernel spins up all drives in parallel, in effect that means they all
spin up at the same time. Which in turn means that you have a large
current spike as all the motors start up, stressing the power supply
and possibly blowing your circuit breaker as 10 machines all come on
after the blackout...

Not spinning up at power on is supposed to avoid this very problem. So
spinning up in parallel might not be a brilliant idea. But if we could
put a few seconds between each spin-up command, that would make sense.

Still, auto-start delay is the best, IMO. 6*ID seconds after power up,
a drives spins up (ID is the SCSI ID). By the time you've finished the
BIOS memory test, most/all of your drives have been spun up.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: faster boots?

[Eric W. Biederman]

Alan Cox <alan@lxorguk.ukuu.org.uk> writes:

> > I find that on heavily scsi systems: one machine spins each of 13 disks 
> > up sequentially.  This makes the initial boot take 3-5 minutes before 
> > init even gets its foot in the door.  If someone made a patch to spin 
> > up scsi disks on the first access, I'd gladly give it a test. ;-)
> 
> Ditto. Especially if it spun them down again when idle for a while. 
> 
> The scsi layer does several things serially it could parallelise. It isnt
> just disk spin up its also things like initialising all scsi controllers
> in parallel.

I'm interested in this kind of thing too.  Though I am wondering if the IDE
layer has the same kind of issues.  I notice because the kernel initialization
time is 50% of my boot time.  I have the kernel loaded 3 seconds from
power on...

My impression is that the time during kernel initialization is either
spent spinning up disks or in a million device probes and things going
on behind the scenes.  I haven't had a chance to look farther though.

Eric

Re: faster boots?

[dean gaudet]

On Thu, 4 Apr 2002, Andrew Morton wrote:

> I guess the greatest benefit would come from reorganising the
> layout of the root filesystem's data and metadata so the
> pagecache prepopulation doesn't have to seek all over the place.

windows xp does this automatically (but it takes a lot of idle time before
it'll start playing with your disk)... search for "bootvis" at
microsoft.com, that tool can force the reorganization to occur.  it's
worth 10%ish there as well (quite noticeable on laptops).  they appear to
reorganize the boot-time files into one big contiguous region.  that's
fetched into their equivalent of the page cache with sequential reads.

it's certainly interesting theory -- trying to do disk layout which is
optimised for particular access patterns... it's kind of a hack to do this
just for boot time, but definitely educational :)

in some ways, the filesystem is the wrong place to do this type of
activity -- you could approach the problem as a block layer device between
the fs and the hardware which maintains statistics on access patterns and
moves blocks around to optimise access time -- which lets you fix all
sorts of seeking problems.  i guess the challenge would be maintaining a
map of logical block number to physical block number.  hmm.  guess that's
kind of hard.

-dean

Re: faster boots?

[Alan Cox]

> My impression is that the time during kernel initialization is either
> spent spinning up disks or in a million device probes and things going
> on behind the scenes.  I haven't had a chance to look farther though.

IDE is less of a problem normally. In the LinuxBIOS case it may become
measurable, and the initial drive spinup has to occur before the firmware
responds. (If I understand it rightly most of the drive firmware is on
the disk). For the spin up/down the IDE folks put it in the drives which 
makes life much simpler

Re: faster boots?

[Alan Cox]

> Be careful here. I did this for a while with a Maxtor 80 GB IDE drive,
> and after a few months, it started making unpleasant noises when
> spinning up (lots of clicking and clacking). I went back to continuous

I can tell you I've had 80Gb Maxtors do that that were always spun up. They
went back and the replacement ones have behaved so far. I don't think its
that related

Re: faster boots?

[Bill Davidsen]

On Thu, 4 Apr 2002 joeja@mindspring.com wrote:

>     Is there some way of making the linux kernel boot faster?  
> 
>     While I know that many people here probably don't reboot there
> machines often, I live in CA where my electrictiy is still high and see
> no reason to keep a machine on that is not in use (i.e. while I sleep or
> am at work). 

  You're dreaming... people here boot their machines a LOT, sometimes even
intentionally. Anyone who runs cetting edge kernels is familiar with boots
;-)
   
>     I tested freebsd on an old P133Mhz/32Meg ram and it booted faster
> with the GENERIC kernel than linux did on a AMD 1200Mhz/512Meg ram,
> which seemed odd.  Linux on that same P133 box also took longer than
> FreeBSD to boot. 

  Bear in mind that the install for BSD often does some setup. However,
booting Linux on a laptop with a configured kernel and initrd file, the
INIT script start in about ten seconds.

>     If I have a machine that does not change from day to day hardware
> wise why when I boot the thing do I need to probe the hardware again and
> again each time?  Would passing more options on the command line help
> like all the addresses and IRQ's of known hardware? 
>     Wouldn't it make sense to store this data on the files system?
> Certainly if something like grub or lilo can figure out how to access a
> file on the drive the kernel could check for a 'defaults' file or
> something to get the default irq's, hardware, interrupts, etc from. 
> Then the kernel could probe these first and if the probe fails proble
> elsewhere for the device. 

  That really should take little time... slow boots are caused by two
things, a generic kernel which has everything possible included, and a
slow init script which not only does many things, but does them serially
and accompanied by many messages. Watching Redhat tap dancing and farting
through line after line of stuff is painful, and with a serial console on
a server, hooked to a 9600bps modem, you can not only get a cup of coffe,
but brew it and drink it as well.

  You should time your boot stages, time to get the kernel running vs.
time to do init, and where does the init script spend its time. If the
kernel boot itself is slow, build a configured kernel, use only what you
need, make most of it modules demand loaded from modules.conf (later), and
you should get boot time down to ten sec or less.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: faster boots?

[Bill Davidsen]

On Thu, 4 Apr 2002, Benjamin LaHaise wrote:

> I find that on heavily scsi systems: one machine spins each of 13 disks 
> up sequentially.  This makes the initial boot take 3-5 minutes before 
> init even gets its foot in the door.  If someone made a patch to spin 
> up scsi disks on the first access, I'd gladly give it a test. ;-)

  Look at the specs for startup current. Multiply by 13. That's why they
spin up one at a time, many drives draw far more current getting up to
speed, and doing all of them at once can be an issue.

  The initial spin is usually done by the BIOS, if you use RAID and have
the drives spin down when idle (assuming they are ever idle), they can
start all at once. I have seen that pop the little breaker in a power
supply, although it was poorly configured (a manager told me 200w was
fine, 350 was overkill). Four SCSI in a tower, get a query and reset.

  That said, I believe you can tune that in some controllers, but I don't
have a model or example handy. Most of my larger machines don't boot very
often, so it's not an issue for me.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: faster boots?

[Mauricio Nuñez]

Try UPX unstable to compress the kernel. The decompression routine is faster 
than actual kernel routine.

http://upx.sf.net

Mauricio Nuñez
mnunez@maxmedia.cl

Re: faster boots?

[Richard Gooch]

Alan Cox writes:
> > Be careful here. I did this for a while with a Maxtor 80 GB IDE drive,
> > and after a few months, it started making unpleasant noises when
> > spinning up (lots of clicking and clacking). I went back to continuous
> 
> I can tell you I've had 80Gb Maxtors do that that were always spun
> up. They went back and the replacement ones have behaved so far. I
> don't think its that related

Well, one year on, after continuous spinning, I still don't hear
clicking and clacking.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: faster boots?

[Jeremy Jackson]

It may be interesting to note that SGI's XFS has a
"realtime section" or some such, which has no metadata,
only data blocks.  Perhaps DMAPI has methods for moving
files around on a live fs.  If you know what you want in there.

Jeremy

----- Original Message -----
From: "dean gaudet" <dean-list-linux-kernel@arctic.org>
To: "Andrew Morton" <akpm@zip.com.au>
Cc: <joeja@mindspring.com>; <linux-kernel@vger.kernel.org>
Sent: Thursday, April 04, 2002 11:45 PM
Subject: Re: faster boots?


> On Thu, 4 Apr 2002, Andrew Morton wrote:
>
> > I guess the greatest benefit would come from reorganising the
> > layout of the root filesystem's data and metadata so the
> > pagecache prepopulation doesn't have to seek all over the place.
>
> windows xp does this automatically (but it takes a lot of idle time before
> it'll start playing with your disk)... search for "bootvis" at
> microsoft.com, that tool can force the reorganization to occur.  it's
> worth 10%ish there as well (quite noticeable on laptops).  they appear to
> reorganize the boot-time files into one big contiguous region.  that's
> fetched into their equivalent of the page cache with sequential reads.
>
> it's certainly interesting theory -- trying to do disk layout which is
> optimised for particular access patterns... it's kind of a hack to do this
> just for boot time, but definitely educational :)
>
> in some ways, the filesystem is the wrong place to do this type of
> activity -- you could approach the problem as a block layer device between
> the fs and the hardware which maintains statistics on access patterns and
> moves blocks around to optimise access time -- which lets you fix all
> sorts of seeking problems.  i guess the challenge would be maintaining a
> map of logical block number to physical block number.  hmm.  guess that's
> kind of hard.
>
> -dean
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: faster boots?

[Mark H. Wood]

On Thu, 4 Apr 2002 joeja@mindspring.com wrote:
[snip]
> If I have a machine that does not change from day to day hardware
> wise why when I boot the thing do I need to probe the hardware again
> and again each time?  Would passing more options on the command line
> help like all the addresses and IRQ's of known hardware?
> Wouldn't it make sense to store this data on the files system? Certainly
> if something like grub or lilo can figure out how to access a file on
> the drive the kernel could check for a 'defaults' file or something to
> get the default irq's, hardware, interrupts, etc from.  Then the kernel
> could probe these first and if the probe fails proble elsewhere for the
> device.

Eww, it sounds like all that unnecessary and problematic hardware info.
that MS Windows saves in the Registry, instead of doing the sensible
thing by asking the hardware.  Please don't go there.

-- 
Mark H. Wood, Lead System Programmer   mwood@IUPUI.Edu
MS Windows *is* user-friendly, but only for certain values of "user".

Re: faster boots?

[Benjamin LaHaise]

On Fri, Apr 05, 2002 at 08:04:28AM -0500, Bill Davidsen wrote:
> On Thu, 4 Apr 2002, Benjamin LaHaise wrote:
> 
> > I find that on heavily scsi systems: one machine spins each of 13 disks 
> > up sequentially.  This makes the initial boot take 3-5 minutes before 
> > init even gets its foot in the door.  If someone made a patch to spin 
> > up scsi disks on the first access, I'd gladly give it a test. ;-)
> 
>   Look at the specs for startup current. Multiply by 13. That's why they
> spin up one at a time, many drives draw far more current getting up to
> speed, and doing all of them at once can be an issue.

Really?  I didn't know that.  :-P

Seriously, only 2 of the disks need to be spun up to start the system, 
so I'd rather be able to login and have only processes which need to 
access the disks that aren't ready yet wait.  Hence the comment about 
spinning disks up on first access...

Also, the system has a ~1200W power supply, so I think it can spin more 
than 1 disk up at a time.  Again, defaulting to 1 is right and good, but 
making it tunable would be better.

		-ben
-- 
"A man with a bass just walked in,
 and he's putting it down
 on the floor."

Re: faster boots?

[Itai Nahshon]

On Friday 05 April 2002 15:49 pm, Alan Cox wrote:
> > Be careful here. I did this for a while with a Maxtor 80 GB IDE drive,
> > and after a few months, it started making unpleasant noises when
> > spinning up (lots of clicking and clacking). I went back to continuous
>
> I can tell you I've had 80Gb Maxtors do that that were always spun up. They
> went back and the replacement ones have behaved so far. I don't think its
> that related
>

That should all on a per-disk configuration. I would like a different,
configurable  spin-down timeout (optionally never) for each disk.

A required feature IMHO: there should _never_ be dirty blocks
for disks that are not spinning.

Maybe it is possible to internally remount RO file systems that
have data on the disk before spinning down. Should not be much
of a problem if the file system is mounted with noatime and there
aren't any files open for writing nor directory operations in progress.

I'd rather have open() blocking and not fill all the memory with
dirty buffers that need to wait for spin-up to complete before they
can be written.

Swap areas, direct access to block device. loop devices and
raid partitions may be more difficult but you get the idea...

-- Itai

Re: faster boots?

[Benjamin LaHaise]

On Sat, Apr 06, 2002 at 02:02:36AM +0300, Itai Nahshon wrote:
> A required feature IMHO: there should _never_ be dirty blocks
> for disks that are not spinning.

Never make assertions like that: on my laptop, I want *lots* of 
dirty blocks held in memory while the disk isn't spinning.  Keeping 
RAM powered is much less costly than spinning the disk up.

		-ben
-- 
"A man with a bass just walked in,
 and he's putting it down
 on the floor."

Re: faster boots?

[Itai Nahshon]

On Saturday 06 April 2002 02:07 am, Benjamin LaHaise wrote:
> On Sat, Apr 06, 2002 at 02:02:36AM +0300, Itai Nahshon wrote:
> > A required feature IMHO: there should _never_ be dirty blocks
> > for disks that are not spinning.
>
> Never make assertions like that: on my laptop, I want *lots* of
> dirty blocks held in memory while the disk isn't spinning.  Keeping
> RAM powered is much less costly than spinning the disk up.
>
> 		-ben

I figure that if there are dirty blocks that belong to files that you
want to keep, they must be flushed at some time, probably on the
next sync(). On "normal" systems that's likely to happen in less
than a minute.

I admit that what I had in mind was medium-large systems with
multiple disks where some of the disks have very little activity
or small systems where there is really zero disk activity for
a long time.

I'm curios, how much work can you accomplish on your laptop
without any disk access (but you still need to save files - keeping
them in buffers until it's time to actually write them).

-- Itai

Re: faster boots?

[Benjamin LaHaise]

On Sat, Apr 06, 2002 at 03:07:05AM +0300, Itai Nahshon wrote:
> I'm curios, how much work can you accomplish on your laptop
> without any disk access (but you still need to save files - keeping
> them in buffers until it's time to actually write them).

Doesn't everyone prime the cache on their laptop?

		-ben
-- 
"A man with a bass just walked in,
 and he's putting it down
 on the floor."

Re: faster boots?

[Pavel Machek]

Hi!
> I'm curios, how much work can you accomplish on your laptop
> without any disk access (but you still need to save files - keeping
> them in buffers until it's time to actually write them).

Debugging session (emacs/gcc/gdb) for half an hour with disks stopped is
easy to accomplish.
								Pavel
-- 
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.

Re: faster boots?

[Itai Nahshon]

On Sunday 07 April 2002 17:42 pm, Pavel Machek wrote:
> Hi!
>
> > I'm curios, how much work can you accomplish on your laptop
> > without any disk access (but you still need to save files - keeping
> > them in buffers until it's time to actually write them).
>
> Debugging session (emacs/gcc/gdb) for half an hour with disks stopped is
> easy to accomplish.
> 								Pavel

My suggestion was: there should _never_ be dirty blocks for disks that
are not spinning. Flush all dirty buffers before spinning down, and spin-up
on any operation that writes to the disk (and block that operation).

The opposite to that (which I do not like) processes create as many
dirty buffers as they want and disk spins up only on sync() or when
the system is starving for usable memory.

An aletrnate ides (more drastic) is that fle systems can mount internally
read-only when a disk is spinned-down. Means - you cannot spin
down when there is a file handle open for writing. Other than this there
are advantages.

I don't see how any of these will stop you from doing emacs/gcc/gdb
with the disk stopped, as long as you are not trying to write to the
disk or read from files that are not cached.

-- Itai

Re: faster boots?

[Richard Gooch]

Itai Nahshon writes:
> On Sunday 07 April 2002 17:42 pm, Pavel Machek wrote:
> > Hi!
> >
> > > I'm curios, how much work can you accomplish on your laptop
> > > without any disk access (but you still need to save files - keeping
> > > them in buffers until it's time to actually write them).
> >
> > Debugging session (emacs/gcc/gdb) for half an hour with disks stopped is
> > easy to accomplish.
> > 								Pavel
> 
> My suggestion was: there should _never_ be dirty blocks for disks
> that are not spinning. Flush all dirty buffers before spinning down,

This is fine, and I wish the block I/O layer was smarter and could
handle spin/idle management itself, rather than telling the drive to
spin down after inactivity. That way, when the bio layer sees
inactivity, it can tell the drive to go to sleep, but first it can
flush any remaining buffers.

> and spin-up on any operation that writes to the disk (and block that
> operation).

Absolutely not! I don't want my writes to spin up the drive.

> The opposite to that (which I do not like) processes create as many
> dirty buffers as they want and disk spins up only on sync() or when
> the system is starving for usable memory.

Maybe you don't like that, but many of us with laptops prefer that
behaviour. And for many reasons, it is definately the correct
behaviour.

> An aletrnate ides (more drastic) is that fle systems can mount
> internally read-only when a disk is spinned-down. Means - you cannot
> spin down when there is a file handle open for writing. Other than
> this there are advantages.

Undesirable behaviour.

> I don't see how any of these will stop you from doing emacs/gcc/gdb
> with the disk stopped, as long as you are not trying to write to the
> disk or read from files that are not cached.

But I *want* to write while the drive is spun down. And leave it spun
down until the system is RAM starved (or some threshold is reached).

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: faster boots?

[Andrew Morton]

Richard Gooch wrote:
> 
> But I *want* to write while the drive is spun down. And leave it spun
> down until the system is RAM starved (or some threshold is reached).
> 

Yes.  The desirable behaviour for laptops is to defer writes
for a very long time, or until the user says "sync".

Mechanisms need to be put in place so that if there are pending
writes and the disk happens to be spun up for a read, we take
advantage of that spinup to push out the pending writes at
the same time.

This behaviour should be all be enabled by a special "laptop mode"
switch.

There's nothing particularly hard in all this...  I'll do a 2.5
version at some stage.

-

Re: faster boots?

[Andre Hedrick]

Well they are there if they were not deleted by the 2.5 maintainer.
If they were, then feel free to copy and credit the work from 2.4 once I
complete the infrastructure.

Cheers,

Andre Hedrick
LAD Storage Consulting Group

On Sun, 7 Apr 2002, Andrew Morton wrote:

> Richard Gooch wrote:
> > 
> > But I *want* to write while the drive is spun down. And leave it spun
> > down until the system is RAM starved (or some threshold is reached).
> > 
> 
> Yes.  The desirable behaviour for laptops is to defer writes
> for a very long time, or until the user says "sync".
> 
> Mechanisms need to be put in place so that if there are pending
> writes and the disk happens to be spun up for a read, we take
> advantage of that spinup to push out the pending writes at
> the same time.
> 
> This behaviour should be all be enabled by a special "laptop mode"
> switch.
> 
> There's nothing particularly hard in all this...  I'll do a 2.5
> version at some stage.
> 
> -
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: faster boots?

[Oliver Neukum]

> > and spin-up on any operation that writes to the disk (and block that
> > operation).
>
> Absolutely not! I don't want my writes to spin up the drive.

Even if you sync ?

> > The opposite to that (which I do not like) processes create as many
> > dirty buffers as they want and disk spins up only on sync() or when
> > the system is starving for usable memory.
>
> Maybe you don't like that, but many of us with laptops prefer that
> behaviour. And for many reasons, it is definately the correct
> behaviour.

You are definitely right. I'd even wish for swapping out stuff and doing
a drastic read ahead before spinning down.

> > An aletrnate ides (more drastic) is that fle systems can mount
> > internally read-only when a disk is spinned-down. Means - you cannot
> > spin down when there is a file handle open for writing. Other than
> > this there are advantages.
>
> Undesirable behaviour.

Absolutely. One disk is typical and log files will be open.

	Regards
		Oliver

Re: faster boots?

[Helge Hafting]

Itai Nahshon wrote:
> 
> On Sunday 07 April 2002 17:42 pm, Pavel Machek wrote:
> > Hi!
> >
> > > I'm curios, how much work can you accomplish on your laptop
> > > without any disk access (but you still need to save files - keeping
> > > them in buffers until it's time to actually write them).
> >
> > Debugging session (emacs/gcc/gdb) for half an hour with disks stopped is
> > easy to accomplish.
> >                                                               Pavel
> 
> My suggestion was: there should _never_ be dirty blocks for disks that
> are not spinning. 

Why not?  Are you afraid that the spun-down disk won't
start the next time it is needed?

Helge Hafting

Re: faster boots?

[Pavel Machek]

Hi!

> > > I'm curios, how much work can you accomplish on your laptop
> > > without any disk access (but you still need to save files - keeping
> > > them in buffers until it's time to actually write them).
> >
> > Debugging session (emacs/gcc/gdb) for half an hour with disks stopped is
> > easy to accomplish.
> 
> My suggestion was: there should _never_ be dirty blocks for disks that
> are not spinning. Flush all dirty buffers before spinning down, and spin-up
> on any operation that writes to the disk (and block that operation).

And your suggestion is useless because disk will be up all the time on
the system...

> The opposite to that (which I do not like) processes create as many
> dirty buffers as they want and disk spins up only on sync() or when
> the system is starving for usable memory.

...which is usefull, OTOH. If you have laptop with battery and
suspend-to-disk capability in BIOS when battery goes low, you can
cache for *long* time.

> An aletrnate ides (more drastic) is that fle systems can mount internally
> read-only when a disk is spinned-down. Means - you cannot spin
> down when there is a file handle open for writing. Other than this there
> are advantages.

Hah, so you can never spindown because /var/log/syslog is opened for
writing.

								Pavel
-- 
Casualities in World Trade Center: ~3k dead inside the building,
cryptography in U.S.A. and free speech in Czech Republic.

Re: faster boots?

[Pavel Machek]

Hi!
> Richard Gooch wrote:
> > 
> > But I *want* to write while the drive is spun down. And leave it spun
> > down until the system is RAM starved (or some threshold is reached).
> > 
> 
> Yes.  The desirable behaviour for laptops is to defer writes
> for a very long time, or until the user says "sync".
> 
> Mechanisms need to be put in place so that if there are pending
> writes and the disk happens to be spun up for a read, we take
> advantage of that spinup to push out the pending writes at
> the same time.
> 
> This behaviour should be all be enabled by a special "laptop mode"
> switch.
> 
> There's nothing particularly hard in all this...  I'll do a 2.5
> version at some stage.

Well, noflushd already seems to work pretty well ;-). But I see kernel
support may be required for SCSI.
							Pavel
-- 
Casualities in World Trade Center: ~3k dead inside the building,
cryptography in U.S.A. and free speech in Czech Republic.

Re: faster boots?

[Rogier Wolff]

Itai Nahshon wrote:
> On Saturday 06 April 2002 02:07 am, Benjamin LaHaise wrote:
> > On Sat, Apr 06, 2002 at 02:02:36AM +0300, Itai Nahshon wrote:
> > > A required feature IMHO: there should _never_ be dirty blocks
> > > for disks that are not spinning.
> >
> > Never make assertions like that: on my laptop, I want *lots* of
> > dirty blocks held in memory while the disk isn't spinning.  Keeping
> > RAM powered is much less costly than spinning the disk up.
> >
> > 		-ben
> 
> I figure that if there are dirty blocks that belong to files that you
> want to keep, they must be flushed at some time, probably on the
> next sync(). On "normal" systems that's likely to happen in less
> than a minute.
> 
> I admit that what I had in mind was medium-large systems with
> multiple disks where some of the disks have very little activity
> or small systems where there is really zero disk activity for
> a long time.
> 
> I'm curios, how much work can you accomplish on your laptop
> without any disk access (but you still need to save files - keeping
> them in buffers until it's time to actually write them).

On a laptop you can decide to "trust" the drive to spin up, just
because the benefits outweigh the drawbacks.

The benefit may be: Battery life becomes 8 hours instead of 4. That
might mean that you get 4 hours of extra work done while travelling at
$100 per hour.... 

Just editing source-code using VI, or reading docs can leave your 
disk completely idle for hours at a time. 

Debugging an app, compiling testing, recompiling can leave your disk
idle provided you accept dirty blocks in the buffer cache...


			Roger. 

-- 
** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2137555 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
* There are old pilots, and there are bold pilots. 
* There are also old, bald pilots. 

Re: faster boots?

[Rogier Wolff]

Itai Nahshon wrote:
> On Sunday 07 April 2002 17:42 pm, Pavel Machek wrote:
> > Hi!
> >
> > > I'm curios, how much work can you accomplish on your laptop
> > > without any disk access (but you still need to save files - keeping
> > > them in buffers until it's time to actually write them).
> >
> > Debugging session (emacs/gcc/gdb) for half an hour with disks stopped is
> > easy to accomplish.
> > 								Pavel
> 
> My suggestion was: there should _never_ be dirty blocks for disks that
> are not spinning. Flush all dirty buffers before spinning down, and spin-up
> on any operation that writes to the disk (and block that operation).
> 
> The opposite to that (which I do not like) processes create as many
> dirty buffers as they want and disk spins up only on sync() or when
> the system is starving for usable memory.
> 
> An aletrnate ides (more drastic) is that fle systems can mount internally
> read-only when a disk is spinned-down. Means - you cannot spin
> down when there is a file handle open for writing. Other than this there
> are advantages.

Actually, you can allow the spindown with open write-filehandles. You
have to remember to spin it up and remount r/w when activity happens.

I'd really like my systems to remount RO if ilde for long times. I
have a few systems that occasionally stop unexpectedly, and not having
to fsck the non-active filesystems would save a lot of time.

(I've decided I dislike reiser: It ate two of my sources, because I
crashed the machine before the sources hit the disk. But the metadata
had been updated to indicate that the overwritten source was in a
different spot on the disk than the old sources. Having to go back to
an older version is less bad than losing the source altogether. 

Yes, I could turn on data-loggin. Yes, I could type "sync" before
insmodding the new driver.)

			Roger. 

-- 
** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2137555 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
* There are old pilots, and there are bold pilots. 
* There are also old, bald pilots. 

Re: faster boots?

[Rogier Wolff]

Helge Hafting wrote:
> Itai Nahshon wrote:
> > 
> > On Sunday 07 April 2002 17:42 pm, Pavel Machek wrote:
> > > Hi!
> > >
> > > > I'm curios, how much work can you accomplish on your laptop
> > > > without any disk access (but you still need to save files - keeping
> > > > them in buffers until it's time to actually write them).
> > >
> > > Debugging session (emacs/gcc/gdb) for half an hour with disks stopped is
> > > easy to accomplish.
> > >                                                               Pavel
> > 
> > My suggestion was: there should _never_ be dirty blocks for disks that
> > are not spinning. 
> 
> Why not?  Are you afraid that the spun-down disk won't
> start the next time it is needed?

I think he is. The problem is: then it's almost always "too late"
anyway: What are you going to do with a system that suddenly doesn't
have a disk anymore? 

Suppose you enforce that, and first try to spin up the disk before
allowing teh write to continue. Suppose that the spin up fails: 

	vi: EIO writing source.c

Now to recover from this situation you may want to writ the source to 
a floppy then. So in another VC you type:

	mount /dev/fd0 /mnt/floppy
	IO error reading "mount". 

Remember the main disk couldn't be spun up....

This is almost no worse than the write going into the buffer cache,
and the appliction not knowing something went wrong...

			Roger. 

-- 
** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2137555 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
* There are old pilots, and there are bold pilots. 
* There are also old, bald pilots. 

Re: faster boots?

[Bill Davidsen]

On Mon, 8 Apr 2002, Helge Hafting wrote:

> Why not?  Are you afraid that the spun-down disk won't
> start the next time it is needed?

  More to the point, afraid the hardware and o/s aren't perfectly
reliable, the battery life isn't infinite in any mode, etc, etc.

  There are many activities which have a large people time to bytes
written ratio, and I can appraciate that for many things you want a write
every N minutes if there is even one dirty buffer, where N is the
increment of person time you are willing to risk having to recreate.

  The tone of your posting implies that the risk of failure is low and it
is paranoid to worry that the disk will not spin. Obviously the previous
poster values think time highly, and I don't believe that concern should
be devalued, even by implication. 

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: faster boots?

[Bill Davidsen]

On Sun, 7 Apr 2002, Richard Gooch wrote:


> But I *want* to write while the drive is spun down. And leave it spun
> down until the system is RAM starved (or some threshold is reached).

  The threshold I hit is how much think time I want to risk. I have no
problem spinning down the drive after inactivity, but the idea of
investing several hours making little changes in a program or proposal
document and then maybe losing them... batteries are just not that
expensive.

  Looking at the blinking of my disk light, it *appears* that by tuning
bdflush you can get a user selected time between sweeps, and use hdparm to
spin down (I assume your laptop doesn't use SCSI!) after a minute or so.
Tuning bdflush somewhat relies on using either -aa or mainline VM, perhaps
version as well. You should be able to only flush every N minutes with any
of them, but the meaning of /sys/vm/bdflush seems somewhat dependent of
code behind it. Still, you should be able to get the behaviour you want
without making it manditory for everyone.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: faster boots?

[Martin Dalecki]

Richard Gooch wrote:
> Oliver Neukum writes:
> 
>>>>and spin-up on any operation that writes to the disk (and block that
>>>>operation).
>>>
>>>Absolutely not! I don't want my writes to spin up the drive.
>>
>>Even if you sync ?
> 
> 
> I'm undecided. I think it's good to have a way to let the user force a
> flush, but I don't like the same mechanism being used by applications
> which think they know better. So flushing on sync(2) or f*sync(2) is
> perhaps undesirable. Maybe the way to deal with this is to have:
> 
> - tunable flush time (i.e. how long to wait after a write(2) before
>   flushing, if the drive is currently unspun)
> 
> - tunable dirty pages limit (i.e. how many dirty pages allowed before
>   flushing)
> 
> - tunable "ignore *sync(2)" option. Default value is 0 (don't
>   ignore). When set to 1, ignore all calls to *sync(2).
> 
> So then on my 256 MiB laptop, I'd probably set the flush time to 3
> hours, the dirty page limit to 64 MiB, and ignore *sync(2). I'd write
> a suid-root programme which did:
> 	enable_sync ();
> 	sync ();
> 	disable_sync ();

Quite frankly the spin-up-down-up-down-up behaviour of linux on
notbooks even if I let them entierly alone is to say the leasy annoying...
And noflushd didn't help me a jota on this issue. Second I don't
think that going though this cycle even on desktop systems does
really help the reliability of the wearings of the driver. For
some reaons I wasn't able to find all the 1000 parameters one has
to set before the whole thing does shut properly. Curing this by
settign some "low water mark" for the number of allowed dirty pages
is curing the symptoms - if there are no system activities there
simply should be no chance for some crepping dirty pages if this is
still the case there are just chances that there are simple bugs out
there. kflushd should by no chance flush the caches just in case
without checking whatever there was some activity in an ideal world.

Re: faster boots?

[Jamie Lokier]

Pavel Machek wrote:
> Well, noflushd already seems to work pretty well ;-). But I see kernel
> support may be required for SCSI.

I've had no luck at all with noflushd on my Toshiba Satellite 4070CDT.
It would spin down every few minutes, and then spin up _immediately_,
every time.  I have no idea why.

This was noflushd-2.5-1 (the RPM packaged by Daniel Kobras), on some 2.4
kernel (I don't remember which).  I gave up trying it some months ago.

-- Jamie

Re: faster boots?

[Benjamin LaHaise]

On Mon, Apr 08, 2002 at 05:43:33PM +0100, Jamie Lokier wrote:
> Pavel Machek wrote:
> > Well, noflushd already seems to work pretty well ;-). But I see kernel
> > support may be required for SCSI.
> 
> I've had no luck at all with noflushd on my Toshiba Satellite 4070CDT.
> It would spin down every few minutes, and then spin up _immediately_,
> every time.  I have no idea why.

Were you using the console?  Any activity on ttys causes device inode 
atime/mtime updates which trigger disk spin ups.  The easiest way to 
work around this is to run X while using devpts for the ptys.

		-ben
-- 
"A man with a bass just walked in,
 and he's putting it down
 on the floor."

Re: faster boots?

[Richard Gooch]

Oliver Neukum writes:
> 
> > > and spin-up on any operation that writes to the disk (and block that
> > > operation).
> >
> > Absolutely not! I don't want my writes to spin up the drive.
> 
> Even if you sync ?

I'm undecided. I think it's good to have a way to let the user force a
flush, but I don't like the same mechanism being used by applications
which think they know better. So flushing on sync(2) or f*sync(2) is
perhaps undesirable. Maybe the way to deal with this is to have:

- tunable flush time (i.e. how long to wait after a write(2) before
  flushing, if the drive is currently unspun)

- tunable dirty pages limit (i.e. how many dirty pages allowed before
  flushing)

- tunable "ignore *sync(2)" option. Default value is 0 (don't
  ignore). When set to 1, ignore all calls to *sync(2).

So then on my 256 MiB laptop, I'd probably set the flush time to 3
hours, the dirty page limit to 64 MiB, and ignore *sync(2). I'd write
a suid-root programme which did:
	enable_sync ();
	sync ();
	disable_sync ();

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: faster boots?

[Mark Mielke]

Not really thinking about how hard it would be to implement, I suggest
that the appropriate place for this to be, would be a mount option.

Just as 'noatime', or 'sync', perhaps a 'delaywrite' option would be a
good choice. An advantage of this approach, is that I could make /tmp
be 'delaywrite+journal' in an effort to improve the efficiency of
/tmp, as I could care less what I lost in /tmp between reboots under
extreme situations.

mark


On Sun, Apr 07, 2002 at 06:14:51PM -0700, Andrew Morton wrote:
> Richard Gooch wrote:
> > 
> > But I *want* to write while the drive is spun down. And leave it spun
> > down until the system is RAM starved (or some threshold is reached).
> > 
> 
> Yes.  The desirable behaviour for laptops is to defer writes
> for a very long time, or until the user says "sync".
> 
> Mechanisms need to be put in place so that if there are pending
> writes and the disk happens to be spun up for a read, we take
> advantage of that spinup to push out the pending writes at
> the same time.
> 
> This behaviour should be all be enabled by a special "laptop mode"
> switch.
> 
> There's nothing particularly hard in all this...  I'll do a 2.5
> version at some stage.
> 
> -
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 
mark@mielke.cc/markm@ncf.ca/markm@nortelnetworks.com __________________________
.  .  _  ._  . .   .__    .  . ._. .__ .   . . .__  | Neighbourhood Coder
|\/| |_| |_| |/    |_     |\/|  |  |_  |   |/  |_   | 
|  | | | | \ | \   |__ .  |  | .|. |__ |__ | \ |__  | Ottawa, Ontario, Canada

  One ring to rule them all, one ring to find them, one ring to bring them all
                       and in the darkness bind them...

                           http://mark.mielke.cc/

Re: faster boots?

[Richard Gooch]

Bill Davidsen writes:
> On Sun, 7 Apr 2002, Richard Gooch wrote:
> 
> 
> > But I *want* to write while the drive is spun down. And leave it spun
> > down until the system is RAM starved (or some threshold is reached).
> 
>   The threshold I hit is how much think time I want to risk. I have
> no problem spinning down the drive after inactivity, but the idea of
> investing several hours making little changes in a program or
> proposal document and then maybe losing them... batteries are just
> not that expensive.

It's not $$$ I'm concerned about. It's mass. I don't want to carry
around a pile of batteries. I like to travel light, whether when
flying or spending a day at a conference. Being able to walk around
all day with your laptop on your shoulder, but not getting sore, is
*really* nice. It gives you more flexibility. No need to run back to
the hotel after the last session to drop of the ball-and-chain and
then run to the restaurant where people are meeting up.

And when flying, I just travel with my super lightweight laptop bag
and a backpack. I can stuff a week's worth of clothes in there, and
it's still not that heavy. But I don't have room for a pile of extra
batteries, nor do I want to add to the burden on my shoulders. I don't
want to carry an extra bag, because I'd have to check in it, which
would mean:
- I have to wait to pick up my luggage (*if* it arrives at all),
  rather than walking out and feeling smug as I pass all the lusers
  waiting for their luggage
- I have to deal with the creative ways in which animals^Wluggage
  handlers can damage or destroy anything
- I have to find a trolley to help carry all the crap I'm carrying
- I have to obtain coins to pay for the trolley, if I'm unlucky enough
  to be stuck at an airport where the bastards are so cheap as to
  charge you money for the damn trolley. I'm sure they've already
  gotten their pound of my flesh via the airport tax/surcharge/
  improvement fee/mandatory gouge/handrail polishing donation/
  or whatever.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: faster boots?

[Rene Rebe]

Hi.

On: Mon, 8 Apr 2002 13:08:49 -0400,
    Mark Mielke <mark@mark.mielke.cc> wrote:
> Not really thinking about how hard it would be to implement, I suggest
> that the appropriate place for this to be, would be a mount option.
> 
> Just as 'noatime', or 'sync', perhaps a 'delaywrite' option would be a
> good choice. An advantage of this approach, is that I could make /tmp
> be 'delaywrite+journal' in an effort to improve the efficiency of
> /tmp, as I could care less what I lost in /tmp between reboots under
> extreme situations.

Normally /tmp gets "rm -rf"ed in most dists startup scripts
anyway. /var is for local state data ...

But I also would like such options to make power-saving on Laptops
easier (But I would use it for all partitions ...). I also tried to
make my disks spin-down - but I never got this to work nicely (disks
run far too often). Stuff like delayed write would be really nice.

> mark

k33p h4ck1n6
  René

--  
René Rebe (Registered Linux user: #248718 <http://counter.li.org>)

eMail:    rene.rebe@gmx.net
          rene@rocklinux.org

Homepage: http://drocklinux.dyndns.org/rene/

Anyone sending unwanted advertising e-mail to this address will be
charged $25 for network traffic and computing time. By extracting my
address from this message or its header, you agree to these terms.

Re: faster boots?

[G . Sumner Hayes]

On Mon, Apr 08, 2002 at 07:49:19PM +0200, Rene Rebe wrote:
> On: Mon, 8 Apr 2002 13:08:49 -0400,
>     Mark Mielke <mark@mark.mielke.cc> wrote:
> > Just as 'noatime', or 'sync', perhaps a 'delaywrite' option would be a
> > good choice. An advantage of this approach, is that I could make /tmp
> > be 'delaywrite+journal' in an effort to improve the efficiency of
> > /tmp, as I could care less what I lost in /tmp between reboots under
> > extreme situations.
[SNIP]
> But I also would like such options to make power-saving on Laptops
> easier 

And it's not just for laptop power savings.  People putting together quiet
PCs (for dedicated sound work or just because the PC is in the bedroom)
would like to keep the disks spun down as much as possible, too.

The best I've gotten is ext2(not ext3)+noflushd+noatime+short spindown
time on the drives, plus dead crond and atd and a minimal set of
processes running.  I have a Linux box as a stereo component.

  Sumner

Re: faster boots?

[Bill Davidsen]

On Mon, 8 Apr 2002, Richard Gooch wrote:

> Bill Davidsen writes:
> > On Sun, 7 Apr 2002, Richard Gooch wrote:
> > 
> > 
> > > But I *want* to write while the drive is spun down. And leave it spun
> > > down until the system is RAM starved (or some threshold is reached).
> > 
> >   The threshold I hit is how much think time I want to risk. I have
> > no problem spinning down the drive after inactivity, but the idea of
> > investing several hours making little changes in a program or
> > proposal document and then maybe losing them... batteries are just
> > not that expensive.
> 
> It's not $$$ I'm concerned about. It's mass.

  The "I" in my posting referred to my personal preference which is safety
over what to me is a minor inconvenience.

  After looking at disk accesses for a while I *think* diddling bdflush
parameters will prevent disk writes for quite a while if you don't do
reads of uncached data. So far I'm just catting /proc/partitions once a
minute and doing a diff to the previous. looks like a write every ten
minutes or so, what I set in bdflush, probably of syslog mumbling, since
the system is relatively quiescent at the moment.

  Does anyone have a thought on power consumption of flash chips? I have a
20MB compact flash I use as an auxilary backup for critical stuff, "just
in case" and I bet I could put enough on a 64MB to keep the hard drive
spun down for hours, if I were interested in doing so.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: faster boots?

[David Lang]

watch out for the write cycle limits of your flash. they're pretty low
power (at least compared to anything mechanical) but if you're not careful
you can go through their write capability pretty fast.

David Lang



 On Mon, 8 Apr 2002, Bill Davidsen wrote:

> Date: Mon, 8 Apr 2002 14:31:41 -0400 (EDT)
> From: Bill Davidsen <davidsen@tmr.com>
> To: Richard Gooch <rgooch@ras.ucalgary.ca>
> Cc: Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
> Subject: Re: faster boots?
>
> On Mon, 8 Apr 2002, Richard Gooch wrote:
>
> > Bill Davidsen writes:
> > > On Sun, 7 Apr 2002, Richard Gooch wrote:
> > >
> > >
> > > > But I *want* to write while the drive is spun down. And leave it spun
> > > > down until the system is RAM starved (or some threshold is reached).
> > >
> > >   The threshold I hit is how much think time I want to risk. I have
> > > no problem spinning down the drive after inactivity, but the idea of
> > > investing several hours making little changes in a program or
> > > proposal document and then maybe losing them... batteries are just
> > > not that expensive.
> >
> > It's not $$$ I'm concerned about. It's mass.
>
>   The "I" in my posting referred to my personal preference which is safety
> over what to me is a minor inconvenience.
>
>   After looking at disk accesses for a while I *think* diddling bdflush
> parameters will prevent disk writes for quite a while if you don't do
> reads of uncached data. So far I'm just catting /proc/partitions once a
> minute and doing a diff to the previous. looks like a write every ten
> minutes or so, what I set in bdflush, probably of syslog mumbling, since
> the system is relatively quiescent at the moment.
>
>   Does anyone have a thought on power consumption of flash chips? I have a
> 20MB compact flash I use as an auxilary backup for critical stuff, "just
> in case" and I bet I could put enough on a 64MB to keep the hard drive
> spun down for hours, if I were interested in doing so.
>
> --
> bill davidsen <davidsen@tmr.com>
>   CTO, TMR Associates, Inc
> Doing interesting things with little computers since 1979.
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: faster boots?

[Richard B. Johnson]

On Mon, 8 Apr 2002, David Lang wrote:

> watch out for the write cycle limits of your flash. they're pretty low
> power (at least compared to anything mechanical) but if you're not careful
> you can go through their write capability pretty fast.
> 
> David Lang
> 
> 
>

Upon boot, you can mount a ram-disk off from /tmp. That will reduce
the activity when using the usual editors, vi, vim, emacs, and pico,
which all create temp files on /tmp.


Cheers,
Dick Johnson

Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).

                 Windows-2000/Professional isn't.

Re: faster boots?

[David Lang]

On Mon, 8 Apr 2002, Richard B. Johnson wrote:

> On Mon, 8 Apr 2002, David Lang wrote:
>
> > watch out for the write cycle limits of your flash. they're pretty low
> > power (at least compared to anything mechanical) but if you're not careful
> > you can go through their write capability pretty fast.
> >
> > David Lang
> >
> >
> >
>
> Upon boot, you can mount a ram-disk off from /tmp. That will reduce
> the activity when using the usual editors, vi, vim, emacs, and pico,
> which all create temp files on /tmp.

yes, you also need to mount the flash with noatime. even then you want to
be careful about things like autosave.

David Lang

Re: faster boots?

[Richard B. Johnson]

On Mon, 8 Apr 2002, David Lang wrote:

> On Mon, 8 Apr 2002, Richard B. Johnson wrote:
> 
> > On Mon, 8 Apr 2002, David Lang wrote:
> >
> > > watch out for the write cycle limits of your flash. they're pretty low
> > > power (at least compared to anything mechanical) but if you're not careful
> > > you can go through their write capability pretty fast.
> > >
> > > David Lang
> > >
> > >
> > >
> >
> > Upon boot, you can mount a ram-disk off from /tmp. That will reduce
> > the activity when using the usual editors, vi, vim, emacs, and pico,
> > which all create temp files on /tmp.
> 
> yes, you also need to mount the flash with noatime. even then you want to
> be careful about things like autosave.
> 
> David Lang

I'm not talking about flash. I'm talking about creating a RAM-Disk
upon startup, using RAM, and mounting it off from /tmp.

Script started on Mon Apr  8 15:22:47 2002
# mke2fs /dev/ram 4096
mke2fs 1.19, 13-Jul-2000 for EXT2 FS 0.5b, 95/08/09
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
1024 inodes, 4096 blocks
204 blocks (4.98%) reserved for the super user
First data block=1
1 block group
8192 blocks per group, 8192 fragments per group
1024 inodes per group

Writing inode tables: 0/1 done                            
Writing superblocks and filesystem accounting information: done
# mount /dev/ram /tmp
# df
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/sdb1             16603376   4390520  11369448  28% /
/dev/sdc1              6356624   3113264   2920456  52% /alt
/dev/sdc3              2253284   1383916    754908  65% /home/users
/dev/sda1              1048272    282192    766080  27% /dos/drive_C
/dev/sda5              1046224    181280    864944  17% /dos/drive_D
/dev/ram1                 3963        13      3746   0% /tmp
# ls -la /tmp
total 17
drwxr-xr-x   3 root     root         1024 Apr  8 15:23 .
drwxr-xr-x  24 root     root         4096 Apr  8 11:00 ..
drwxr-xr-x   2 root     root        12288 Apr  8 15:23 lost+found
# rmdir /tmp/lost+found
# cp * /tmp
cp: cdrom.root: omitting directory
cp: platinum-rel-V1.05: omitting directory
cp: platinum.2000: omitting directory
cp: platinum.old: omitting directory
cp: platinum.saved: omitting directory
cp: tar.gz: omitting directory
# df
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/sdb1             16603376   4390520  11369448  28% /
/dev/sdc1              6356624   3113264   2920456  52% /alt
/dev/sdc3              2253284   1383916    754908  65% /home/users
/dev/sda1              1048272    282192    766080  27% /dos/drive_C
/dev/sda5              1046224    181280    864944  17% /dos/drive_D
/dev/ram1                 3963      1915      1844  51% /tmp
# ls /tmp
2Alex.mail	 config.gz   friday.pdf      reboot.tar.gz    timer.tar.gz
Allocate.sh	 copy.iso    get_cdmusic.sh  reject_probe.sh  unix2dos.c
ME.TXT		 cp.iso      gettime.sh      resume.txt       windows.bad
WhereIs.PowerPC  dos2unix.c  grok.sh	     security.sh      zzz.c
backup.cdr	 eject.c     junk.c	     sockperf.c
backup.iso	 fifo.c      launch.tar      test_ramdisk.sh
bomb.tar.gz	 fpu.c	     new.xinitrc     testpoll.c
buffer.sh	 fpu_test.c  platinum.spc    testrd.sh
# umount /tmp
# exit
exit

Script done on Mon Apr  8 15:24:39 2002


Cheers,
Dick Johnson

Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).

                 Windows-2000/Professional isn't.

Re: faster boots?

[Pavel Machek]

Hi!

> > > Well, noflushd already seems to work pretty well ;-). But I see kernel
> > > support may be required for SCSI.
> > 
> > I've had no luck at all with noflushd on my Toshiba Satellite 4070CDT.
> > It would spin down every few minutes, and then spin up _immediately_,
> > every time.  I have no idea why.
> 
> Were you using the console?  Any activity on ttys causes device inode 
> atime/mtime updates which trigger disk spin ups.  The easiest way to 
> work around this is to run X while using devpts for the ptys.

Noflushd does not spin up disks for write activity.
								Pavel
-- 
Casualities in World Trade Center: ~3k dead inside the building,
cryptography in U.S.A. and free speech in Czech Republic.

Re: faster boots?

[Jamie Lokier]

Benjamin LaHaise wrote:
> > > Well, noflushd already seems to work pretty well ;-). But I see kernel
> > > support may be required for SCSI.
> > 
> > I've had no luck at all with noflushd on my Toshiba Satellite 4070CDT.
> > It would spin down every few minutes, and then spin up _immediately_,
> > every time.  I have no idea why.
> 
> Were you using the console?  Any activity on ttys causes device inode 
> atime/mtime updates which trigger disk spin ups.  The easiest way to 
> work around this is to run X while using devpts for the ptys.

I was using X, nodiratime on all /dev/hda mounts.  My friend who has the
small VAIO with a Crusoe chip also reports the same problem: noflushd
doesn't work with 2.4 kernels (versions that we tried), and the problem
is the same: it spins down and then spins up immediately afterward.

That just gave me a small idea: maybe the log message to say "disk spun
down" was triggering a write to disk which spun the disk up again... :-)
My syslog.conf is quite clear about having - in front of the paths for
/var/log/messages and /var/log/kernel though, so syslogd should not be
calling fsync.

If someone knows that noflushd has been since version 2.5 for
2.4.current (or 2.5.current) kernels, then I'll give it another spin as
it were, and report the results.  I'd much prefer "proper" kernel
support as is being discussed in this thread though, i.e. writeout of
dirty pages prior to spin down, that sort of thing.  noflushd always
seemed to me a rather questionable hack.

cheers,
-- Jamie

Re: faster boots?

[Pavel Machek]

Hi!

> > > > Well, noflushd already seems to work pretty well ;-). But I see kernel
> > > > support may be required for SCSI.
> > > 
> > > I've had no luck at all with noflushd on my Toshiba Satellite 4070CDT.
> > > It would spin down every few minutes, and then spin up _immediately_,
> > > every time.  I have no idea why.
> > 
> > Were you using the console?  Any activity on ttys causes device inode 
> > atime/mtime updates which trigger disk spin ups.  The easiest way to 
> > work around this is to run X while using devpts for the ptys.
> 
> I was using X, nodiratime on all /dev/hda mounts.  My friend who has the
> small VAIO with a Crusoe chip also reports the same problem: noflushd
> doesn't work with 2.4 kernels (versions that we tried), and the problem
> is the same: it spins down and then spins up immediately afterward.

It works for me, 2.4.18 on HP omnibook xe3.

You may want to watch /proc/stats to see if it is read or write
activity that wakes disk up.

									Pavel
-- 
Casualities in World Trade Center: ~3k dead inside the building,
cryptography in U.S.A. and free speech in Czech Republic.

Re: faster boots?

[Mike Touloumtzis]

On Fri, Apr 05, 2002 at 07:56:08AM -0500, Bill Davidsen wrote:
> 
> That really should take little time... slow boots are caused by two
> things, a generic kernel which has everything possible included, and a
> slow init script which not only does many things, but does them serially
> and accompanied by many messages. Watching Redhat tap dancing and farting
> through line after line of stuff is painful, and with a serial console on
> a server, hooked to a 9600bps modem, you can not only get a cup of coffe,
> but brew it and drink it as well.

The worst part is when you forget to remove "-C" from the init script
fsck on a machine with a 9600bps serial console.  After a power hit
one machine in our machine closet took 2 hours to come back; finally
figured out it spent most of that time waiting to write progress bars
and twirling batons to /dev/ttyS0.

miket

Re: faster boots?

[Jamie Lokier]

Pavel Machek wrote:
> > > > I've had no luck at all with noflushd on my Toshiba Satellite 4070CDT.
> > > > It would spin down every few minutes, and then spin up _immediately_,
> > > > every time.  I have no idea why.
> > > 
> > > Were you using the console?  Any activity on ttys causes device inode 
> > > atime/mtime updates which trigger disk spin ups.  The easiest way to 
> > > work around this is to run X while using devpts for the ptys.
> > 
> > I was using X, nodiratime on all /dev/hda mounts.  My friend who has the
> > small VAIO with a Crusoe chip also reports the same problem: noflushd
> > doesn't work with 2.4 kernels (versions that we tried), and the problem
> > is the same: it spins down and then spins up immediately afterward.
> 
> It works for me, 2.4.18 on HP omnibook xe3.
> 
> You may want to watch /proc/stats to see if it is read or write
> activity that wakes disk up.

It's write activity, due to atime updates.  I was using nodiratime, but
that's not good enough because every time an executable is run a load of
things are accessed.

I found it interesting that some write activity happens almost
immediately after the access -- and noflushd is connected in some way.
If I do this:

    while :; do cat /proc/stat; sleep 1; done

Then I see a few writes have occurred at nearly every iteration.  I
think that is due to the atime updates, because using "noatime" there
are no writes at most iterations.

But more interesting: I only see those few-per-second atime writes while
noflushd is running.  If I kill noflushd then they go away.

So, noflushd triggers some kind of regular write activity.  Either
killing noflushd, or mounting with "noatime", makes it go away.

I don't like "noatime" because some programs monitor
/var/spool/mail/jamie's atime to decide if there is any new mail.  But I
am using it now anyway.

With "noatime", I find the disk is able to spin down for 20 seconds.  A
record :-)  But not a very useful one.

When the disk spins up, I see both read and write activity at the same
time.  Of course I have no idea what files are triggering the spin up.
(And atime is switched off so I can't use that as a guide!)

I am a bit surprised that "noatime" makes a difference -- I thought that
if noflushd spun down a disk, then pending inode writes should be
delayed until a read or excess memory pressure forces a spin up.

So: "noatime" is definitely required, to spin the disk down for more
than an instant.  But even that is not good enough.  I have 192MB RAM,
btw.  Is that enough to expect longer spin down times than 20s?

-- Jamie

Re: faster boots?

[Pavel Machek]

Hi!

> > It works for me, 2.4.18 on HP omnibook xe3.
> > 
> > You may want to watch /proc/stats to see if it is read or write
> > activity that wakes disk up.
> 
> It's write activity, due to atime updates.  I was using nodiratime, but
> that's not good enough because every time an executable is run a load of
> things are accessed.
> 
> I found it interesting that some write activity happens almost
> immediately after the access -- and noflushd is connected in some way.
> If I do this:
> 
>     while :; do cat /proc/stat; sleep 1; done
> 
> Then I see a few writes have occurred at nearly every iteration.  I
> think that is due to the atime updates, because using "noatime" there
> are no writes at most iterations.

Well, that's no problem. noflushd stops kflushd, so it should work
even with atime. [It works for me with atimes!]

> But more interesting: I only see those few-per-second atime writes while
> noflushd is running.  If I kill noflushd then they go away.

?

> I am a bit surprised that "noatime" makes a difference -- I thought that
> if noflushd spun down a disk, then pending inode writes should be
> delayed until a read or excess memory pressure forces a spin up.

Tha'ts idea behind noflushd. I don't know why it does not work for
you.

> So: "noatime" is definitely required, to spin the disk down for more
> than an instant.  But even that is not good enough.  I have 192MB RAM,
> btw.  Is that enough to expect longer spin down times than 20s?

With noflushd, noatime should not and is not required.

								Pavel
-- 
Casualities in World Trade Center: ~3k dead inside the building,
cryptography in U.S.A. and free speech in Czech Republic.

Re: faster boots?

[Jamie Lokier]

Pavel Machek wrote:
> > If I do this:
> > 
> >     while :; do cat /proc/stat; sleep 1; done
> > 
> > Then I see a few writes have occurred at nearly every iteration.  I
> > think that is due to the atime updates, because using "noatime" there
> > are no writes at most iterations.
> 
> Well, that's no problem. noflushd stops kflushd, so it should work
> even with atime. [It works for me with atimes!]
> 
> > But more interesting: I only see those few-per-second atime writes while
> > noflushd is running.  If I kill noflushd then they go away.
> 
> ?

Another curious thing: even if the regular writes were caused by atime
updates, there is no reason for them to be flushed every second, is
there?

Yet the hard disk light flashes about once per second when (a) running
the above shell line and (b) running noflushd, and (c) _not_ using
"noatime" (just "nodiratime").  (Remove any of those and it stops).  And
/proc/stat shows writes happening.

This is on ext3.  I wonder if journalling is causing a problem.  Pavel,
are you running ext3?

The kernel, by the way, is vanilla 2.4.18.

-- Jamie

Re: faster boots?

[Pavel Machek]

Hi!

> > >     while :; do cat /proc/stat; sleep 1; done
> > > 
> > > Then I see a few writes have occurred at nearly every iteration.  I
> > > think that is due to the atime updates, because using "noatime" there
> > > are no writes at most iterations.
> > 
> > Well, that's no problem. noflushd stops kflushd, so it should work
> > even with atime. [It works for me with atimes!]
> > 
> > > But more interesting: I only see those few-per-second atime writes while
> > > noflushd is running.  If I kill noflushd then they go away.
> > 
> > ?
> 
> Another curious thing: even if the regular writes were caused by atime
> updates, there is no reason for them to be flushed every second, is
> there?
> 
> Yet the hard disk light flashes about once per second when (a) running
> the above shell line and (b) running noflushd, and (c) _not_ using
> "noatime" (just "nodiratime").  (Remove any of those and it stops).  And
> /proc/stat shows writes happening.
> 
> This is on ext3.  I wonder if journalling is causing a problem.  Pavel,
> are you running ext3?

No, plain old ext2. Yep, ext3 could have some strange interactions
with noflushd.
								Pavel
-- 
Casualities in World Trade Center: ~3k dead inside the building,
cryptography in U.S.A. and free speech in Czech Republic.

Re: faster boots?

[Philipp Matthias Hahn]

Moin!

On Fri, Apr 12, 2002 at 03:29:18PM +0100, Jamie Lokier wrote:
> This is on ext3.  I wonder if journalling is causing a problem.  Pavel,
> are you running ext3?

ext3 ignored kflushd and writes directly to the hard-disk every 30 s.
Reading the ext2-devel mailing list at
http://lists.sourceforge.net/lists/listinfo/ext2-devel, the problem is
known and being worked on. Until then, don't use journalling on disk,
you want to spin down.

BYtE
Philipp
-- 
Carl von Ossietzky Universitaet Oldenburg
Betriebssysteme und verteilte Systeme

Re: faster boots?

[Stevie O]

At 02:08 PM 4/5/2002 -0500, Mark H. Wood wrote:
>Eww, it sounds like all that unnecessary and problematic hardware info.
>that MS Windows saves in the Registry, instead of doing the sensible
>thing by asking the hardware.  Please don't go there.

Amen! While it seems logical/intelligent on the surface (it makes it easy to specify consistent module parameters to drivers), it falls victim to something I call the Windows Effect, which is (IMO) the primary reason people still use Windows:

Windows is easy to use, as long as it stops working.
As soon as something breaks, however, you're screwed.

-

I knew someone who had to reinstall Win2K pro on her machine because some registry setting for some driver got screwed up, and DHCP refused to start (thus preventing her from getting an IP for her cable modem).

As I told her -- if she had Linux running, I could have fixed the problem easily (assuming that problem could have happened in the first place).


--
Stevie-O

Real programmers use COPY CON PROGRAM.EXE

Re: faster boots?

[Joe]

Thanks all,
    several people had suggested this.

    as it turns out one thing that was slowing my boot down was actually the
framebuffer.  When the kernel boots and it switches to mode 0x317 (maybe other
modes or all did not try) it would actually take a second or two to switch to
the new mode.  This is on a fast machine so I imagine that on a slower machine
it may take longer.

    also it seems that ide2 probe would take long as well.  maybe less than 1/2
second but it would be noticable so adding ide2=noprobe helped as well.  this
may be that there is no ide2 and it has to time out.  I guess the time out is
very small.

    also the cdrom probe takes a little time as well...

    removeing all these unnecessary drivers has speed it up some more as well.
it seems it now boots in about 1/2 the time.

Joe

> > So what is the best way in Linux to figure out what you can remove from the
> > kernel to make it smaller and boot hopefully faster on low end machines?
>
> Say "M" to everything that isnt your root file system or directly dependant
> on it. The whole "build a custom kernel" thing is mostly a red herring.

Re: faster boots?

[Itai Nahshon]

On Friday 05 April 2002 16:21 pm, willy tarreau wrote:
> > It's normally done with hdparm spindown when idle
> > options...
>
> only supported on IDE devices.
>
> Willy

There was a scsi_idle patch for spinning down scsi disks.
IIRC it was before 2.0 (for the 1.1.x or 1.2.x kernels). Anyone
know what happened with that patch and why it hasn't got into
the official kernel?
-- Itai

RE: faster boots?

[Torrey Hoffman]

[... cc's trimmed, I guess everyone is on the list... ]

I also would like to speed up boots.  On the systems I'm 
responsible for (set top boxes hooked up to television 
sets) the slowest part of the boot is mounting the five reiser 
filesystems.  I'm using the busybox init and the "boot scripts" 
are actually a single compiled program.

Could mounting filesystems be parallelized effectively?

Three of the five filesystems on are mounted read-only, so I 
don't know what reiserfs is doing at mount that takes so long.
It used to be faster when we were using 2.2.19 with the reiser
3.5 patches...  

Does ext3 mount faster?

Torrey Hoffman
thoffman@arnor.net
torrey.hoffman@myrio.com

Re: faster boots?

[willy tarreau]

Bill Davidsen <davidsen@tmr.com> wrote : 

> There is a scsi-idle program of some age
> floating around, I haven't looked at that but
> I know it exists.

I have it on my server, and used it with the
associated patch for kernel 2.2. The problem
is not to spin the disk down, which is handled
in userspace, but to automatically spin it up
upon any access. Thus the patch. I have not
ported it to 2.4 because it wasn't trivial and
I didn't have time for it. But I may one day I
will be bored with my server's noise and I'll
do it. If you want to give it a try, you can
download it from one of my old 2.2 trees :

http://www-miaif.lip6.fr/willy/linux-patches/server-kernels/2.2.19-wt3/H-I9-2.2.19-wt3-wtstuff/scsi-idle-2.2.15.diff

Cheers,
Willy


___________________________________________________________
Do You Yahoo!? -- Une adresse @yahoo.fr gratuite et en français !
Yahoo! Mail : http://fr.mail.yahoo.com

Re: faster boots?

[Bill Davidsen]

On Fri, 5 Apr 2002, [iso-8859-1] willy tarreau wrote:

> > It's normally done with hdparm spindown when idle
> > options...
> 
> only supported on IDE devices.

I didn't have a machine to test, that's not clear from the man page:

       Some  options (eg. -r for SCSI) may not work with old ker-
       nels as necessary ioctl()'s were not supported.

       Although this utility is intended primarily for  use  with
       (E)IDE  hard disk devices, several of the options are also
       valid (and permitted) for use with SCSI hard disk  devices
       and MFM/RLL hard disks with XT interfaces.
and

       -S     Set  the  standby (spindown) timeout for the drive.
              This value is used by the drive  to  determine  how
              long to wait (with no disk activity) before turning
              off the spindle motor to save  power.   Under  such
              circumstances,  the  drive  may  take as long as 30
              seconds to respond to  a  subsequent  disk  access,
              though  most drives are much quicker.  The encoding
              of the timeout value is somewhat peculiar.  A value
              of  zero means "off".  Values from 1 to 240 specify
              multiples of 5 seconds, for timeouts from 5 seconds
              to 20 minutes.  Values from 241 to 251 specify from
              1 to 11 units of 30 minutes, for timeouts  from  30
              minutes  to  5.5 hours.  A value of 252 signifies a
              timeout of 21 minutes, 253  sets  a  vendor-defined
              timeout,  and 255 is interpreted as 21 minutes plus
              15 seconds.

so I thought this might be helpful for someone with the problem to
consider. I'll be building a suitable machine in the next few days, so I
will try it with current hdparm, just for my own edification.

There is a scsi-idle program of some age floating around, I haven't looked
at that but I know it exists. As I recall it used the sg interface, but
that's purely what I remember from the days when a 600MB Maxtor made me a
king. I went through a few years ago and backed them all up to one CD
each...

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: faster boots?

[Luigi Genoni]

What is really noisy is the scsi boot on some server, when you have 4 or
more disks and some other stuff, like on some of my sparc64.
Sometimes It can happen that a reboot asks some minute just for all scsi
and fibbre stuff. If you have MC^2 and some logical volume around on it,
it can be really noisy, especially if you are in a hurry, as always
when it comes to production servers :(.

On desktop PC kernel boot is hardly a problem, the real
slowdown are the init script, but then every distribution
has its own (slower or faster).

On Fri, 5 Apr 2002, Helge Hafting wrote:

> joeja@mindspring.com wrote:
> >
> > Think pre init scripts....
> >
> > no apache was install on this machine, no iptables scripts, etc.
> >
> > I'm actually talking about the time from where Linux spits out all this crap about probing irq's, ide drive found with dma etc.  That kind of stuff.
>
> Compile your own kernel with drivers only for hardware you
> actually have and use during boot.  Omit all other drivers.  That gets
> rid
> of a lot of probing, and the time from "kernel loaded" to "starting
> init"
> gets quite short.  Also, some drivers lets you specify irq's etc.
> on the kernel command line - that may avoid further probing.
>
> Drivers for things not needed in the kernel boot process (cdrom, floppy,
> ethernet, etc.) can be made modular.  That avoids time-consuming cd
> spinup
> and probing of non-boot devices.
>
> The kernel boot is only a few seconds on my office machine.  The bios
> boot is one long delay, the initscripts another.
>
> Helge Hafting
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: faster boots?

[willy tarreau]

> It's normally done with hdparm spindown when idle
> options...

only supported on IDE devices.

Willy


___________________________________________________________
Do You Yahoo!? -- Une adresse @yahoo.fr gratuite et en français !
Yahoo! Mail : http://fr.mail.yahoo.com

Re: faster boots?

[Bill Davidsen]

On Fri, 5 Apr 2002, [iso-8859-1] willy tarreau wrote:

> I had a patch for this in my 2.2 kernel tree,
> originally from wingel@ctrl-c.liu.se, but I
> didn't have time to port it to 2.4.

  It's normally done with hdparm spindown when idle options...

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: faster boots?

[Thomas 'Dent' Mirlacher]

improving kernel bootup time:

	what about saving the parameters for successfully loaded modules?
	(like the IRQs and other stuff which gets autodetected.)
	- and when those modules are loaded for the second time, the
		saves values will be used, without the need to do
		autoprobing (don't know how much time that will
		save, but it should save quite some time for people
		having all the drivers in the kernel, but no matching
		hardware)
		- this will only work if there are no static inits,
			but everything's modularized (which should
			be in the works?)
	- also you need to have a parameter to reprobe the whole modules


- this would also give a nice template for building a customized kernel
(well again for people who are not really used to do things like that)

	just my $0.02

		tm

-- 
in some way i do, and in some way i don't.

Re: faster boots?

[Andi Kleen]

dean gaudet <dean-list-linux-kernel@arctic.org> writes:
> 
> in some ways, the filesystem is the wrong place to do this type of
> activity -- you could approach the problem as a block layer device between
> the fs and the hardware which maintains statistics on access patterns and
> moves blocks around to optimise access time -- which lets you fix all
> sorts of seeking problems.  i guess the challenge would be maintaining a
> map of logical block number to physical block number.  hmm.  guess that's
> kind of hard.

LVM does this already (including the statistics). I guess you could write
a pass to reorganize PEs and make sure that a bootup time the PEs 
are always read completely even when they contain multiple files.

-Andi

Re: faster boots?

[willy tarreau]

> Ditto. Especially if it spun them down again
> when idle for a while.


I had a patch for this in my 2.2 kernel tree,
originally from wingel@ctrl-c.liu.se, but I
didn't have time to port it to 2.4. I've been
using it happily for about 2 years on the NFS
server which is too close to my bedroom. The
only annoying side of the problem is that when
you incidentelly do an NFS access and the disks
are down, you have to wait a few minutes before
they spin up, especially with raid. I once
failed a CD burning session from the NFS server
because doing something else simultaneously
woke other disks up, which hung the NFS server
until they were ready.

But it's definitely interesting, and missing
in 2.4 IMHO.

If you want to try it for your 2.2 tree, I
can send you the patch.

Cheers,
Willy


___________________________________________________________
Do You Yahoo!? -- Une adresse @yahoo.fr gratuite et en français !
Yahoo! Mail : http://fr.mail.yahoo.com

Re: faster boots?

[Helge Hafting]

joeja@mindspring.com wrote:
> 
> Think pre init scripts....
> 
> no apache was install on this machine, no iptables scripts, etc.
> 
> I'm actually talking about the time from where Linux spits out all this crap about probing irq's, ide drive found with dma etc.  That kind of stuff.

Compile your own kernel with drivers only for hardware you
actually have and use during boot.  Omit all other drivers.  That gets
rid
of a lot of probing, and the time from "kernel loaded" to "starting
init"
gets quite short.  Also, some drivers lets you specify irq's etc.
on the kernel command line - that may avoid further probing.

Drivers for things not needed in the kernel boot process (cdrom, floppy, 
ethernet, etc.) can be made modular.  That avoids time-consuming cd
spinup
and probing of non-boot devices.

The kernel boot is only a few seconds on my office machine.  The bios
boot is one long delay, the initscripts another.

Helge Hafting