kjournald and disk sleeping

kjournald and disk sleeping

[Marcos Dione]

	Hi. first of all, I'm not suscribed to the mailing list, so cc to
me in the replies. thanks. and I'm running 2.4.10.

	what I'm doing is to try to put the disks to sleep at night, or
when I'm not using the machine. I found what proceses to shutdown, mainly
those that do things from time to time, like the MTA. then I send a STOP
signal to kupdated. so far, so good. that works.

	then I switched to ext3 and kjournald started to appear on the
processes list. and it commits the transactions very often. I know I can
set the commit interval to a high value, but both I don't know exactly
how, and I think that it's not the solution I need. sending STOP signals
to kjournald doesn't work, it seems to ignore them. what can I do?

	One thing I thought: how is this supposed to work on laptops? can
they be suspended? a question related to this one: I also have ACPI turned
on and APM turned off. how can I switch to stanby states? is there a way?
again, how does it works on laptops?

	remember to cc to me. bye.

-- 
"y, bueno, yo soy muy ilogico. lo que pasa es que ustedes me toman
demasiado en serio"
                                          --JLB

Re: kjournald and disk sleeping

[Marcos Dione]

On Mon, 22 Oct 2001, Andrew Morton wrote:

> Yes, this is a bit of a problem - it's probably atime updates,
> things which write to inodes, etc.  A commit will be forced within
> five seconds of this happening.

	Reading journal.c I guessed that kjournald flushes thing *even if
it doesn't have things to flush*. I guess that from commit_timeout and
the comments on the thread process, but I can be wrong.

	YFI, I issue a /bin/sync before I put the disk to sleep.

-- 
"y, bueno, yo soy muy ilogico. lo que pasa es que ustedes me toman
demasiado en serio"
                                          --JLB

Re: kjournald and disk sleeping

[Marcos Dione]

On Mon, 22 Oct 2001, Andreas Dilger wrote:

> Don't use 2.4.10 Linus kernel with ext3.  It is bad.  Use a newer kernel,
> or -ac kernel instead.

	ok, anyways I'll switch to 2.4.12 this evening.

> Hmm.  I have a laptop running with all ext3 filesystems, and it has no
> problems spinning down the disk.  I have not done anything to increase
> the flush interval of kjournald.  It may be that kjournald is writing
> to disk because you have things which are trying to write to disk.

	uh, and is there any way to find out which processes are doing so?
and now that I think, my current kernel is *not* patched with ext3 (my
previous 2.4.8 had it) but I have reiser support *as module* with no
reiser filesystem mounted. and not only kjornald is there, it has nowhere
to write. hmm, I'll see again thin evening and tell you tomorrow.

> > then I send a STOP signal to kupdated
>
> Well, this is a sure sign that you are getting disk write requests.
> Note that it is very dangerous to do this.  Instead, you should give it
> a long (but finite) interval so that you at least get some data written
> to disk instead of none at all.

	but I really want the machine to be suspended, in a very small
way: I mean, I know that suspending is not complete, but spinning down the
disks and entering in a conservative state both system and cpu will save
energy. stopping kupdated and switching off swap is for that.

> If you change the commit interval and run in journaled-data mode, and have
> a long interval to kupdated, then ext3 _should_ buffer all of your I/O in
> memory until the journal is full.  This is much safer than just turning off
> kupdated, since you WILL get things written to disk if there have been enough
> changes to fill the journal, so you have an upper limit of a few MB of data
> that can be lost if it never flushes to disk.

	mmm, anyways, I'm supossedly not writing to the disk. shutting
down commits should not loose anything because there's nothing to loose.

-- 
"y, bueno, yo soy muy ilogico. lo que pasa es que ustedes me toman
demasiado en serio"
                                          --JLB

Re: kjournald and disk sleeping

[Andrew Morton]

Marcos Dione wrote:
> 
>         Hi. first of all, I'm not suscribed to the mailing list, so cc to
> me in the replies. thanks. and I'm running 2.4.10.
> 
>         what I'm doing is to try to put the disks to sleep at night, or
> when I'm not using the machine. I found what proceses to shutdown, mainly
> those that do things from time to time, like the MTA. then I send a STOP
> signal to kupdated. so far, so good. that works.
> 
>         then I switched to ext3 and kjournald started to appear on the
> processes list. and it commits the transactions very often.

Yes, this is a bit of a problem - it's probably atime updates,
things which write to inodes, etc.  A commit will be forced within
five seconds of this happening.

> I know I can set the commit interval to a high value, but both I don't
> know exactly how, and I think that it's not the solution I need.

That is certainly a simple way of addressing the problem, and
it does work.  You'll need to edit fs/jbd/journal.c and change the `5'
in this line:

        journal->j_commit_interval = (HZ * 5);

to 3600 or whatever.  I'd agree that this user interface could be
improved :)  Probably a field in the journal superblock.

The result of this change is that you could lose up to an hour's work
after a crash rather than up to five seconds worth.  You can manually
force a commit at any time by running /bin/sync.

Probably the best way of addressing all of this is teach ext3 to
look at the kupdate writeback interval from /proc/sys/vm/bdflush.
Users can then set the value in there to, say, one hour and it
should all just work.

 
-

Re: kjournald and disk sleeping

[Andreas Dilger]

On Oct 22, 2001  14:29 -0300, Marcos Dione wrote:
> 	Hi. first of all, I'm not suscribed to the mailing list, so cc to
> me in the replies. thanks. and I'm running 2.4.10.

Don't use 2.4.10 Linus kernel with ext3.  It is bad.  Use a newer kernel,
or -ac kernel instead.

> 	then I switched to ext3 and kjournald started to appear on the
> processes list. and it commits the transactions very often. I know I can
> set the commit interval to a high value, but both I don't know exactly
> how, and I think that it's not the solution I need. sending STOP signals
> to kjournald doesn't work, it seems to ignore them. what can I do?

Hmm.  I have a laptop running with all ext3 filesystems, and it has no
problems spinning down the disk.  I have not done anything to increase
the flush interval of kjournald.  It may be that kjournald is writing
to disk because you have things which are trying to write to disk.

> then I send a STOP signal to kupdated

Well, this is a sure sign that you are getting disk write requests.
Note that it is very dangerous to do this.  Instead, you should give it
a long (but finite) interval so that you at least get some data written
to disk instead of none at all.

I have all of the filesystems on my laptop mounted noatime (this breaks
/tmp auto-cleanup, but oh well) and I have moved all of the entries from
/etc/cron.hourly to /etc/cron.daily, as they are not so critical for me.

If you want, you can still increase the kjournald flush interval by editing
fs/jbd/journal.c:journal_init_common().  Granted, this is not elegant, and
it _should_ be configurable somewhere, but it isn't yet.

If you change the commit interval and run in journaled-data mode, and have
a long interval to kupdated, then ext3 _should_ buffer all of your I/O in
memory until the journal is full.  This is much safer than just turning off
kupdated, since you WILL get things written to disk if there have been enough
changes to fill the journal, so you have an upper limit of a few MB of data
that can be lost if it never flushes to disk.

Cheers, Andreas
--
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

Re: kjournald and disk sleeping

[Andrew Morton]

Marcos Dione wrote:
> 
> On Mon, 22 Oct 2001, Andrew Morton wrote:
> 
> > Yes, this is a bit of a problem - it's probably atime updates,
> > things which write to inodes, etc.  A commit will be forced within
> > five seconds of this happening.
> 
>         Reading journal.c I guessed that kjournald flushes thing *even if
> it doesn't have things to flush*. I guess that from commit_timeout and
> the comments on the thread process, but I can be wrong.

It's rather convoluted, but a commit fires if:

1: more than approx 1/4 of the journal has been used by the
   current transaction or

2: this transaction has been open for >5 seconds.

And a commit will close off the current transaction, but will *not*
open a new one.  Opening a new transaction will only happen when
new writes come in.

So there should be no commit activity on an fs which isn't being
written to.

You can watch all this happening by building with debug support and
running

	echo 1 > /proc/sys/fs/jbd-debug

You may want to stop syslogd first though - otherwise you get into
this loop where commits create logs and logs create commits.  Tends
to fill your logs up rather boringly.

-

Re: kjournald and disk sleeping

[Stephen C. Tweedie]

Hi,

On Mon, Oct 22, 2001 at 11:28:46AM -0700, Andrew Morton wrote:

> Yes, this is a bit of a problem - it's probably atime updates,

You can mount a filesystem with the "noatime" option, though.  That's
useful on laptops to stop read accesses from spinning up the disk.

Cheers,
 Stephen

Re: kjournald and disk sleeping

[Marcos Dione]

On Mon, 22 Oct 2001, Andreas Dilger wrote:

> Hmm.  I have a laptop running with all ext3 filesystems, and it has no
> problems spinning down the disk.  I have not done anything to increase
> the flush interval of kjournald.  It may be that kjournald is writing
> to disk because you have things which are trying to write to disk.

	indeed, seems that a lot of apps like to access the disk when
they're doing nothing. now I have two approaches: one is to find them all,
the other is to simply stomp almos all the apps, includin servers. tonight
I'll make some tests.

> I have all of the filesystems on my laptop mounted noatime (this breaks
> /tmp auto-cleanup, but oh well) and I have moved all of the entries from
> /etc/cron.hourly to /etc/cron.daily, as they are not so critical for me.

	this may help too, thanks.

-- 
"y, bueno, yo soy muy ilogico. lo que pasa es que ustedes me toman
demasiado en serio"
                                          --JLB

Re: kjournald and disk sleeping

[Pavel Machek]

Hi!

> 	One thing I thought: how is this supposed to work on laptops? can
> they be suspended? a question related to this one: I also have ACPI turned
> on and APM turned off. how can I switch to stanby states? is there a way?
> again, how does it works on laptops?

I'm working on suspend-to-disk, and suspend-to-ram is mostly working, also.
...
								Pavel
-- 
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.

Re: kjournald and disk sleeping

[CaT]

On Thu, Oct 25, 2001 at 04:13:31PM +0000, Pavel Machek wrote:
> Hi!
> 
> > 	One thing I thought: how is this supposed to work on laptops? can
> > they be suspended? a question related to this one: I also have ACPI turned
> > on and APM turned off. how can I switch to stanby states? is there a way?
> > again, how does it works on laptops?
> 
> I'm working on suspend-to-disk, and suspend-to-ram is mostly working, also.
> ...

Sweet.

What's not working with suspend-to-ram? Gateway, in their infinate
wisdon, nuked suspend-to-disk functionality of the bios in the most
recent edition (which they kindly upgraded me to when I put my laptop
in for servicing...). As such I only have suspend-to-ram working and
I'm also interested in playing with ACPI.

-- 
CaT        "As you can expect it's really affecting my sex life. I can't help
           it. Each time my wife initiates sex, these ejaculating hippos keep
           floating through my mind."
                - Mohd. Binatang bin Goncang, Singapore Zoological Gardens

Re: kjournald and disk sleeping

[Pavel Machek]

Hi!

> > > 	One thing I thought: how is this supposed to work on laptops? can
> > > they be suspended? a question related to this one: I also have ACPI turned
> > > on and APM turned off. how can I switch to stanby states? is there a way?
> > > again, how does it works on laptops?
> > 
> > I'm working on suspend-to-disk, and suspend-to-ram is mostly working, also.
> > ...
> 
> Sweet.
> 
> What's not working with suspend-to-ram? Gateway, in their infinate

Patrick Mochel from transmeta. [Or you meant suspend-to-disk?]

> wisdon, nuked suspend-to-disk functionality of the bios in the most
> recent edition (which they kindly upgraded me to when I put my laptop
> in for servicing...). As such I only have suspend-to-ram working and
> I'm also interested in playing with ACPI.

								Pavel
-- 
Casualities in World Trade Center: 6453 dead inside the building,
cryptography in U.S.A. and free speech in Czech Republic.

Re: kjournald and disk sleeping

[CaT]

On Fri, Oct 26, 2001 at 11:54:47PM +0200, Pavel Machek wrote:
> > > I'm working on suspend-to-disk, and suspend-to-ram is mostly working, also.
> > > ...
> > 
> > Sweet.
> > 
> > What's not working with suspend-to-ram? Gateway, in their infinate
> 
> Patrick Mochel from transmeta. [Or you meant suspend-to-disk?]

Actually, now that I think about it, both. :)

-- 
CaT        "As you can expect it's really affecting my sex life. I can't help
           it. Each time my wife initiates sex, these ejaculating hippos keep
           floating through my mind."
                - Mohd. Binatang bin Goncang, Singapore Zoological Gardens