Background scanning change on 2.4.6-pre1

Background scanning change on 2.4.6-pre1

[Marcelo Tosatti]

Hi Linus, 


Who did this change to refill_inactive_scan() in 2.4.6-pre1 ? 

        /*
         * When we are background aging, we try to increase the page aging
         * information in the system.
         */
        if (!target)
                maxscan = nr_active_pages >> 4;

This is going to make all pages have age 0 on an idle system after some
time (the old code from Rik which has been replaced by this code tried to 
avoid that)

Could you please explain me the reasoning behind this change ?  

Thanks 

Re: Background scanning change on 2.4.6-pre1

[Marcelo Tosatti]

On Thu, 7 Jun 2001, Linus Torvalds wrote:

> 
> Forgot one comment..
> 
> > > This is going to make all pages have age 0 on an idle system after some
> > > time (the old code from Rik which has been replaced by this code tried to 
> > > avoid that)
> 
> There's another reason why I think the patch may be ok even without any
> added logic: not only does it simplify the code and remove a illogical
> heuristic, but there is nothing that really says that "age 0" is
> necessarily very bad.
> 
> We should strive to keep the active/inactive lists in LRU order anyway, so
> the ordering does tell you something about how recent (and thus how
> important) the page is. Also, it's certainly MUCH preferable to let pages
> age down to zero, than to let pages retain a maximum age over a long time,
> like the old code used to do.
> 
> If, after long periods of inactivity, we start needing fresh pages again,
> it's probably actually an _advantage_ to give the new pages a higher
> relative importance. Caches tend to lose their usefulness over time, and
> if the old cached pages are really relevant, then the new spurt of usage
> will obviously mark them young again.
> 
> And if, after the idle time, the behaviour is different, the old pages
> have appropriately been aged down and won't stand in the way of a new
> cache footprint.
> 
> Do you actually have regular usage that shows the age-down to be a bad
> thing? 

Fill the active list of cache and wait for a while to get the inactive
list full.

When that happens and pressure begins, refill_inactive_scan() from
try_to_free_pages() will not be called because the inactive list is full
(the kernel "thinks" we dont have an inactive shortage). Well, not sure if
this is a bad thing in the end. 

Re: Background scanning change on 2.4.6-pre1

[Linus Torvalds]

On Thu, 7 Jun 2001, Marcelo Tosatti wrote:
> 
> Who did this change to refill_inactive_scan() in 2.4.6-pre1 ? 

I think it was Andreas Dilger..

>         /*
>          * When we are background aging, we try to increase the page aging
>          * information in the system.
>          */
>         if (!target)
>                 maxscan = nr_active_pages >> 4;
> 
> This is going to make all pages have age 0 on an idle system after some
> time (the old code from Rik which has been replaced by this code tried to 
> avoid that)

He posted a nice explanation of how this change made behaviour noticeably
smoother, and how you could actually see the nicer balance between the
active and inactive lists using osview.

The code is not necessarily "correct", but this patch was accompanied by
useful real-life user information.

Now, I think the problem with the old code was that it didn't do _any_
background page aging if "inactive" was large enough. And that really
doesn't make all that much sense. Background page aging is needed to
"sort" the active list, regardless of how many inactive pages there are.

The decision to not do page aging should not be based on the number of
inactive pages, and I think the patch is correct in that sense.

Now, if you were to change the code to something like

	/* background scanning? */
	if (!target) {
		if (atomic_read(page_aging) <= 0)
			return 0;
		maxscan = nr_active_pages >> 4;
	}

and make the "page_aging" be something that goes up when we age stuff up
and goes down when we age it down, and does the proper balancing, THAT
would probably be ok. Then the decision to not age in the background would
be based on whether we have lots of pages getting aged up or not.

Heuristics should make sense, not be "random".

		Linus

Re: Background scanning change on 2.4.6-pre1

[Linus Torvalds]

Forgot one comment..

> > This is going to make all pages have age 0 on an idle system after some
> > time (the old code from Rik which has been replaced by this code tried to 
> > avoid that)

There's another reason why I think the patch may be ok even without any
added logic: not only does it simplify the code and remove a illogical
heuristic, but there is nothing that really says that "age 0" is
necessarily very bad.

We should strive to keep the active/inactive lists in LRU order anyway, so
the ordering does tell you something about how recent (and thus how
important) the page is. Also, it's certainly MUCH preferable to let pages
age down to zero, than to let pages retain a maximum age over a long time,
like the old code used to do.

If, after long periods of inactivity, we start needing fresh pages again,
it's probably actually an _advantage_ to give the new pages a higher
relative importance. Caches tend to lose their usefulness over time, and
if the old cached pages are really relevant, then the new spurt of usage
will obviously mark them young again.

And if, after the idle time, the behaviour is different, the old pages
have appropriately been aged down and won't stand in the way of a new
cache footprint.

Do you actually have regular usage that shows the age-down to be a bad
thing? 

		Linus

Re: Background scanning change on 2.4.6-pre1

[Andreas Dilger]

Linus writes:
> On Thu, 7 Jun 2001, Marcelo Tosatti wrote:
> > Who did this change to refill_inactive_scan() in 2.4.6-pre1 ? 
> 
> I think it was Andreas Dilger..

Definitely NOT.  I don't touch MM stuff.  I do filesystems and LVM only.

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

Re: Background scanning change on 2.4.6-pre1

[Jonathan Morton]

>> > This is going to make all pages have age 0 on an idle system after some
>> > time (the old code from Rik which has been replaced by this code tried to
>> > avoid that)
>
>There's another reason why I think the patch may be ok even without any
>added logic: not only does it simplify the code and remove a illogical
>heuristic, but there is nothing that really says that "age 0" is
>necessarily very bad.

Here's my take on it.  The point of ageing is twofold - to age down pages
that aren't in use, and to age up pages that *are* in use.  So, pages that
are in use will remain with high ages even when background scanning is
being done, and pages that aren't in use will decay to zero age.

I can't see what's wrong with that.  When we need more memory, it's a Very
Good Thing to know that most of the pages in the system haven't been
accessed in yonks - we know exactly which ones we want to throw out first.

--------------------------------------------------------------
from:     Jonathan "Chromatix" Morton
mail:     chromi@cyberspace.org  (not for attachments)

The key to knowledge is not to rely on people to teach you it.

GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS
PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)

Re: Background scanning change on 2.4.6-pre1

[Andreas Dilger]

Marcello writes:
> Who did this change to refill_inactive_scan() in 2.4.6-pre1 ? 
> 
>         /*
>          * When we are background aging, we try to increase the page aging
>          * information in the system.
>          */
>         if (!target)
>                 maxscan = nr_active_pages >> 4;

A quick check in the l-k archives shows this was Zlatko Calusic
<zlatko.calusic@iskon.hr> who submitted the patch.  See

http://marc.theaimsgroup.com/?l=linux-kernel&m=99151955000988&w=4

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

Re: Background scanning change on 2.4.6-pre1

[Rik van Riel]

On Thu, 7 Jun 2001, Linus Torvalds wrote:
> On Thu, 7 Jun 2001, Marcelo Tosatti wrote:

> > time (the old code from Rik which has been replaced by this code tried to 
> > avoid that)
> 
> Now, I think the problem with the old code was that it didn't do _any_
> background page aging if "inactive" was large enough. And that really
> doesn't make all that much sense. Background page aging is needed to
> "sort" the active list, regardless of how many inactive pages there are.

I'll be posting a patch in a few minutes (against 2.4.5-acX, which
was the latest kernel available to me while on holidays with no
net access) which doesn't "roll over" the inactive dirty pages when
we scan the list.

This should make us reclaim the inactive_dirty pages in a much better
LRU order, so this whole background aging limiting stuff becomes close
to moot.

regards,

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/		http://distro.conectiva.com/

Send all your spam to aardvark@nl.linux.org (spam digging piggy)