page pre-swapping + moving it on cache-list

page pre-swapping + moving it on cache-list

[Samium Gromoff]

       Here is an idea i think i stole from Matthew Dillon`s paper.

    Actually it sound more like we take some pages from the near 0 
  age and swapping them out but not throwing them away, but moving them
  from active list to cache. So that we can always throw them away
  at near null cost while shrinking the cache. This is like a
  replacement for swap-cache if i`m right here...

cheers,
 Sam
   

Re: page pre-swapping + moving it on cache-list

[Rik van Riel]

On Thu, 6 Sep 2001, Samium Gromoff wrote:

>        Here is an idea i think i stole from Matthew Dillon`s paper.
>
>     Actually it sound more like we take some pages from the near 0
>   age and swapping them out but not throwing them away, but moving them
>   from active list to cache. So that we can always throw them away
>   at near null cost while shrinking the cache. This is like a
>   replacement for swap-cache if i`m right here...

This is called the "inactive_clean" list in Linux terminology. ;)

Rik
-- 
IA64: a worthy successor to i860.

http://www.surriel.com/		http://distro.conectiva.com/

Send all your spam to aardvark@nl.linux.org (spam digging piggy)

Re: page pre-swapping + moving it on cache-list

[Stephan von Krawczynski]

On Thu, 6 Sep 2001 08:29:03 -0300 (BRST) Rik van Riel <riel@conectiva.com.br>
wrote:

> On Thu, 6 Sep 2001, Samium Gromoff wrote:
> 
> >        Here is an idea i think i stole from Matthew Dillon`s paper.
> >
> >     Actually it sound more like we take some pages from the near 0
> >   age and swapping them out but not throwing them away, but moving them
> >   from active list to cache. So that we can always throw them away
> >   at near null cost while shrinking the cache. This is like a
> >   replacement for swap-cache if i`m right here...
> 
> This is called the "inactive_clean" list in Linux terminology. ;)

What's the use of an inactive_clean list anyway, or the effective difference
between its members and the members of memfree (I suspect such a list from the
output of /proc/meminfo)?
Besides the fact, that the splitting in two lists prevents proper
defragmentation (if you have pages in two lists you are not driven to defrag at
the point where you put them together in one).

Regards,
Stephan

Re: page pre-swapping + moving it on cache-list

[Rik van Riel]

On Thu, 6 Sep 2001, Stephan von Krawczynski wrote:

> What's the use of an inactive_clean list anyway, or the effective
> difference between its members and the members of memfree (I suspect
> such a list from the output of /proc/meminfo)?

Pages on the inactive_clean list contain data. Throwing
away data when you can keep it in memory isn't the smartest
thing.

> Besides the fact, that the splitting in two lists prevents proper
> defragmentation

Patches to fix this thing while still allowing us to keep
the data in memory for most pages are appreciated. Patches
to the universe to give me more than 24 hours a day, too ;)

cheers,

Rik
-- 
IA64: a worthy successor to i860.

http://www.surriel.com/		http://distro.conectiva.com/

Send all your spam to aardvark@nl.linux.org (spam digging piggy)

Re: page pre-swapping + moving it on cache-list

[Alex Bligh - linux-kernel]

Stephan / Rik,

--On Thursday, September 06, 2001 10:05 AM -0300 Rik van Riel 
<riel@conectiva.com.br> wrote:

> Pages on the inactive_clean list contain data. Throwing
> away data when you can keep it in memory isn't the smartest
> thing.
>
>> Besides the fact, that the splitting in two lists prevents proper
>> defragmentation
>
> Patches to fix this thing while still allowing us to keep
> the data in memory for most pages are appreciated.

It's not keeping it two lists that prevents proper
defragmentation. It's having it allocated all around
memory, and never freeing the pages which prevent
coalescence of areas, and thus either
throwing away the data, or garbage collecting, that
prevents defragmentation. We never free these pages
either because they sit in an idle system (with
no memory pressure) in places like
inactive_dirty, or in caches, or, in an active
system, because they are direct_reclaim'd out
of inactive_clean etc., so never get freed [1].

As this kind of garbage collection requires memcpy()
etc., it might be harmless when the system is
idle, but isn't going to be an attractive option when
the system is busy thrashing away (though it might be
possible to hueristically evict awkward pages
preferentially, by aging them more harshly).

But before we go introducing tentacles of the buddy
system into every part of the memory manager, I'd
like to be satisfied we can't attempt to allocate
and free stuff more intelligently in the first place.

I have some more ideas on this I shall code up
tonight.

[1] though as disabling direct_reclaim entirely
    has NO measurable effect on fragmentation, fixing
    it is at best a necessary but not sufficient component
    of a fix). I haven't measured the effect of Roger L's
    patch in total.


--
Alex Bligh

Re: page pre-swapping + moving it on cache-list

[Jonathan Lundell]

At 2:21 PM +0100 2001-09-06, Alex Bligh - linux-kernel wrote:
>It's not keeping it two lists that prevents proper
>defragmentation. It's having it allocated all around
>memory, and never freeing the pages which prevent
>coalescence of areas, and thus either
>throwing away the data, or garbage collecting, that
>prevents defragmentation. We never free these pages
>either because they sit in an idle system (with
>no memory pressure) in places like
>inactive_dirty, or in caches, or, in an active
>system, because they are direct_reclaim'd out
>of inactive_clean etc., so never get freed [1].
>
>As this kind of garbage collection requires memcpy()
>etc., it might be harmless when the system is
>idle, but isn't going to be an attractive option when
>the system is busy thrashing away (though it might be
>possible to hueristically evict awkward pages
>preferentially, by aging them more harshly).

The problem with thrashing, is it not, is that we're not making 
forward progress because we're waiting for swap--that is to say, 
thrashing *is* an idle state of sorts, and so might be an ideal 
opportunity for gc methods that require heavy CPU involvement. It's 
not as if there's anything better to do....
-- 
/Jonathan Lundell.

Re: page pre-swapping + moving it on cache-list

[Helge Hafting]

Jonathan Lundell wrote:

> The problem with thrashing, is it not, is that we're not making
> forward progress because we're waiting for swap--that is to say,
> thrashing *is* an idle state of sorts, and so might be an ideal
> opportunity for gc methods that require heavy CPU involvement. It's
> not as if there's anything better to do....

Note that trashing don't necessarily mean the cpu is free.
It can be very busy:
- deciding what to swap out next
- queuing stuff up for io, merging long elevator queues
- handling io operations, we don't all have busmastering devices

somehow I don't think garbage collection runs will be that fun
in a trashing situation.  Don't these algorithms look all over
your stack & heap for pointers?  That will surely cause lots
of io as all the apps memory is paged in so the gc algorithm
may look at it.  

Helge Hafting

Re: page pre-swapping + moving it on cache-list

[Alex Bligh - linux-kernel]

Helge,

> somehow I don't think garbage collection runs will be that fun
> in a trashing situation.

Quite possibly

> Don't these algorithms look all over
> your stack & heap for pointers? That will surely cause lots
> of io as all the apps memory is paged in so the gc algorithm
> may look at it.

No - it would look through things like the free area
table, the buddy bitmaps, the page table & lists etc., all
of which are, of course, in kernel memory. So while
it may do unfortunate things to the cache, it doesn't
need to touch application memory in order to determine
which pages to twiddle with. Of course twiddling the
pages themselves requires access to them, but if they
are out on disk already they consume (or, if on
InactiveClean, could consume) no physical memory
so are the least of our problems w.r.t. memory
defragmentation.

--
Alex Bligh