From: Daniel Phillips <phillips@bonn-fries.net>
To: Marco Colombo <marco@esi.it>
Cc: Rik van Riel <riel@conectiva.com.br>, <mike_phillips@urscorp.com>,
<linux-kernel@vger.kernel.org>
Subject: Re: VM Requirement Document - v0.0
Date: Wed, 4 Jul 2001 17:03:26 +0200 [thread overview]
Message-ID: <01070417032606.03760@starship> (raw)
In-Reply-To: <Pine.LNX.4.33.0107041033230.4236-100000@Megathlon.ESI>
In-Reply-To: <Pine.LNX.4.33.0107041033230.4236-100000@Megathlon.ESI>
On Wednesday 04 July 2001 11:41, Marco Colombo wrote:
> On Tue, 3 Jul 2001, Daniel Phillips wrote:
> > On Tuesday 03 July 2001 12:33, Marco Colombo wrote:
> > > Oh, yes, since that PAGE_AGE_BG_INTERACTIVE_MINIMUM is applied only
> > > when background aging, maybe it's not enough to keep processes like
> > > updatedb from causing interactive pages to be evicted.
> > > That's why I said we should have another way to detect that kind of
> > > activity... well, the application could just let us know (no need to
> > > embed an autotuning-genetic-page-replacement-optimizer into the
> > > kernel). We should just drop all FS metadata accessed by updatedb,
> > > since we know that's one-shot only, without raising pressure at all.
> >
> > Note that some of updatedb's metadata pages are of the accessed-often
> > kind, e.g., directory blocks and inodes. A blanket low priority on all
> > the pages updatedb touches just won't do.
>
> Remember that the first message was about a laptop. At 4:00AM there's
> no activity but the updatedb one (and the other cron jobs). Simply,
> there's no 'accessed-often' data. Moreover, I'd bet that 90% of the
> metadata touched by updatedb won't be accessed at all in the future.
> Laptop users don't do find /usr/share/terminfo/ so often.
The problem is when you have a directory block, say, that has to stay around
quite a few seconds before dropping into disuse. You sure don't want that
block treated as 'accessed-once'.
The goal here is to get through the updatedb as quickly as possible. Getting
the user's "interactive" programs loaded back in afterwards is a separate,
much more difficult problem IMHO, but no doubt still has a reasonable
solution. I'm not that worried about it, my feeling is: if we fix up the MM
so it doesn't bog down with a lot of pages in cache and, in addition, do
better readahead, interactive performance will be just fine.
> > > Just like
> > > (not that I'm proposing it) putting those "one-shot" pages directly on
> > > the inactive-clean list instead of the active list. How an application
> > > could declare such a behaviour is an open question, of course. Maybe
> > > it's even possible to detect it. And BTW that's really fine tuning.
> > > Evicting an 8 hours old page may be a mistake sometime, but it's never
> > > a *big* mistake.
> >
> > IMHO, updatedb *should* evict all the "interactive" pages that aren't
> > actually doing anything[1]. That way it should run faster, provided of
> > course its accessed-once pages are properly given low priority.
>
> So in the morning you find your Gnome session completely on swap,
> and at the same time a lot of free mem.
>
> > I see three page priority levels:
> >
> > 0 - accessed-never/aged to zero
> > 1 - accessed-once/just loaded
> > 2 - accessed-often
> >
> > with these transitions:
> >
> > 0 -> 1, if a page is accessed
> > 1 -> 2, if a page is accessed a second time
> > 1 -> 0, if a page gets old
> > 2 -> 0, if a page gets old
> >
> > The 0 and 1 level pages are on a fifo queue, the 2 level pages are
> > scanned clock-wise, relying on the age computation[2]. Eviction
> > candidates are taken from the cold end of the 0 level list, unless it is
> > empty, in which case they are taken from the 1 level list. In
> > desperation, eviction candidates are taken from the 2 level list, i.e.,
> > random eviction policy, as opposed to what we do now which is to initiate
> > an emergency scan of the active list for new inactive candidates - rather
> > like calling a quick board meeting when the building is on fire.
>
> Well, it's just aging faster when it's needed. Random evicting is not
> good.
It's better than getting bogged down in scanning latency just at the point
you should be starting new writeouts. Obviously, it's a tradeoff.
> List 2 is ordered by age, and there're always better candidates
> at the end of the list than at the front. The higher the pressure,
> the shorter is the time a page has to rest idle to get at the end of the
> list. But the list *is* ordered.
No, list 2 is randomly ordered. Pages move from the initial trial list to
the active list with 0 temperature, and drop in just behind the one-hand scan
pointer (which we actually implement as the head of the list). After that
they get "aged" up or down as we do now. (New improved terminology: heated
or cooled according to the referenced bit.)
> > Note that the above is only a very slight departure from the current
> > design. And by the way, this is just brainstorming, it hasn't reached the
> > 'proposal' stage yet.
> >
> > [1] It would be nice to have a mechanism whereby the evicted
> > 'interactive' pages are automatically reloaded when updatedb has finished
> > its work. This is a case of scavenging unused disk bandwidth for
> > something useful, i.e., improving the interactive experience.
>
> updatedb doesn't really need all the memory it takes. All it needs is
> a small buffer to sequentially scan all the disk. So we should just
> drop all the pages it references, since we already know they won't be
> referenced again by noone else.
I hope it's clear how the method I'm describing does that.
> > [2] I much prefer the hot/cold terminology over old/young. The latter
> > gets confusing because a 'high' age is 'young'. I'd rather think of a
> > high value as being 'hot'.
>
> True. s/page->age/page->temp/g B-)
Yep.
--
Daniel
next prev parent reply other threads:[~2001-07-04 15:00 UTC|newest]
Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top
2001-06-28 12:20 VM Requirement Document - v0.0 mike_phillips
2001-06-28 12:30 ` Alan Cox
2001-06-28 13:33 ` Tobias Ringstrom
2001-06-28 13:37 ` Alan Cox
2001-06-28 14:04 ` Tobias Ringstrom
2001-06-28 14:14 ` Alan Cox
2001-06-28 14:52 ` Daniel Phillips
2001-06-28 14:39 ` Daniel Phillips
2001-06-28 18:01 ` Marco Colombo
2001-07-02 18:42 ` Rik van Riel
2001-07-03 10:33 ` Marco Colombo
2001-07-03 15:04 ` Daniel Phillips
2001-07-03 18:24 ` Daniel Phillips
2001-07-04 8:12 ` Ari Heitner
2001-07-04 9:41 ` Marco Colombo
2001-07-04 15:03 ` Daniel Phillips [this message]
2001-07-03 18:29 ` Daniel Phillips
2001-07-04 8:32 ` Marco Colombo
2001-07-04 14:44 ` Daniel Phillips
2001-06-28 15:21 ` Jonathan Morton
2001-06-28 16:02 ` Daniel Phillips
-- strict thread matches above, loose matches on Subject: below --
2001-07-05 15:09 mike_phillips
2001-07-05 15:04 Daniel Phillips
[not found] ` <fa.jprli0v.qlofoc@ifi.uio.no>
[not found] ` <fa.e66agbv.hn0u1v@ifi.uio.no>
2001-07-05 1:49 ` Dan Maas
2001-07-05 13:02 ` Daniel Phillips
2001-07-05 14:00 ` Xavier Bestel
2001-07-05 14:51 ` Daniel Phillips
2001-07-05 15:00 ` Xavier Bestel
2001-07-05 15:12 ` Daniel Phillips
2001-07-05 15:12 ` Alan Shutko
[not found] ` <002501c104f4/mnt/sendme701a8c0@morph>
2001-07-09 12:17 ` Pavel Machek
2001-07-12 23:46 ` Daniel Phillips
2001-07-13 21:07 ` Pavel Machek
2001-07-06 19:09 ` Rik van Riel
2001-07-06 21:57 ` Daniel Phillips
2001-07-04 16:08 mike_phillips
[not found] <fa.oqkojpv.3hosb7@ifi.uio.no>
[not found] ` <fa.jpsks3v.1o2gag4@ifi.uio.no>
2001-06-27 0:43 ` Dan Maas
2001-06-27 0:45 ` Mike Castle
2001-06-27 10:50 ` Xavier Bestel
2001-06-27 8:53 Martin Knoblauch
2001-06-27 18:13 ` Rik van Riel
2001-06-28 6:59 ` Martin Knoblauch
2001-06-28 11:27 ` Helge Hafting
2001-06-28 11:54 ` Martin Knoblauch
2001-06-28 12:02 ` Tobias Ringstrom
2001-06-28 12:31 ` Xavier Bestel
2001-06-28 13:05 ` Tobias Ringstrom
2001-06-26 19:58 Jason McMullan
2001-06-26 21:21 ` Rik van Riel
2001-06-26 21:29 ` Jason McMullan
2001-06-26 21:33 ` John Stoffel
2001-06-26 21:42 ` Rik van Riel
2001-06-26 22:21 ` Stefan Hoffmeister
2001-06-26 22:48 ` Jeffrey W. Baker
2001-06-27 0:18 ` Mike Castle
2001-06-28 13:07 ` John Fremlin
2001-06-27 13:36 ` Marco Colombo
2001-06-27 3:55 ` Daniel Phillips
2001-06-27 14:09 ` Pozsar Balazs
2001-06-28 22:47 ` John Fremlin
2001-06-30 15:37 ` Pavel Machek
2001-07-10 10:34 ` David Woodhouse
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=01070417032606.03760@starship \
--to=phillips@bonn-fries.net \
--cc=linux-kernel@vger.kernel.org \
--cc=marco@esi.it \
--cc=mike_phillips@urscorp.com \
--cc=riel@conectiva.com.br \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).