Strange behaviour of swap under 2.4.5-ac15

Strange behaviour of swap under 2.4.5-ac15

[German Gomez Garcia]

	Hello,

	I've running 2.4.5-ac15 for almost a day (22 hours) and I found
some strange behaviour of the kswap, at least it was not present in
2.4.5-ac9. The swap memory increase with time as the cache dedicated
memory also increase, that is swapping process at a very fast rate, even
when no program is getting more memory. Is that the expected behaviour?
	An example, with no process running (just the usual daemons and
none of them getting extra memory) the command:

	free ; sleep 60; free

             total       used       free     shared    buffers     cached
Mem:        513416     393184     120232        364      63276     254576
-/+ buffers/cache:      75332     438084
Swap:       530104      14228     515876

             total       used       free     shared    buffers     cached
Mem:        513416     393192     120224        364      63276     258412
-/+ buffers/cache:      71504     441912
Swap:       530104      18064     512040

	Any idea?

	- german

-------------------------------------------------------------------------
German Gomez Garcia         | "This isn't right.  This isn't even wrong."
<german@piraos.com>         |                         -- Wolfgang Pauli

Re: Strange behaviour of swap under 2.4.5-ac15

[Andrea Arcangeli]

On Mon, Jun 18, 2001 at 12:14:01PM +0200, German Gomez Garcia wrote:
> 	Hello,
> 
> 	I've running 2.4.5-ac15 for almost a day (22 hours) and I found
> some strange behaviour of the kswap, at least it was not present in
> 2.4.5-ac9. The swap memory increase with time as the cache dedicated
> memory also increase, that is swapping process at a very fast rate, even
> when no program is getting more memory. Is that the expected behaviour?
> 	An example, with no process running (just the usual daemons and
> none of them getting extra memory) the command:
> 
> 	free ; sleep 60; free
> 
>              total       used       free     shared    buffers     cached
> Mem:        513416     393184     120232        364      63276     254576
> -/+ buffers/cache:      75332     438084
> Swap:       530104      14228     515876
> 
>              total       used       free     shared    buffers     cached
> Mem:        513416     393192     120224        364      63276     258412
> -/+ buffers/cache:      71504     441912
> Swap:       530104      18064     512040
> 
> 	Any idea?

either apply this patch to 2.4.5ac15:

	ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.5aa3/00_fix-unusable-vm-on-alpha-1

(note it is not an alpha specific bug, it's just that I was triggering
all the time on alpha so I called the patch that way)

or better use 2.4.6pre3aa1:

	ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.6pre3aa1.bz2

If the problem persists let me know thanks.

Andrea

Re: Strange behaviour of swap under 2.4.5-ac15

[German Gomez Garcia]

On Mon, 18 Jun 2001, Andrea Arcangeli wrote:

> On Mon, Jun 18, 2001 at 12:14:01PM +0200, German Gomez Garcia wrote:
> > 	Hello,
> >
> > 	I've running 2.4.5-ac15 for almost a day (22 hours) and I found
> > some strange behaviour of the kswap, at least it was not present in
> > 2.4.5-ac9. The swap memory increase with time as the cache dedicated
> > memory also increase, that is swapping process at a very fast rate, even
> > when no program is getting more memory. Is that the expected behaviour?
> > 	An example, with no process running (just the usual daemons and
> > none of them getting extra memory) the command:
> >
> > 	free ; sleep 60; free
> >
> >              total       used       free     shared    buffers     cached
> > Mem:        513416     393184     120232        364      63276     254576
> > -/+ buffers/cache:      75332     438084
> > Swap:       530104      14228     515876
> >
> >              total       used       free     shared    buffers     cached
> > Mem:        513416     393192     120224        364      63276     258412
> > -/+ buffers/cache:      71504     441912
> > Swap:       530104      18064     512040
> >
> > 	Any idea?
>
> either apply this patch to 2.4.5ac15:
>
> 	ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.5aa3/00_fix-unusable-vm-on-alpha-1
>
> (note it is not an alpha specific bug, it's just that I was triggering
> all the time on alpha so I called the patch that way)

	It doesn't fix it, swapped memory and cache memory increase at a
rate of about 40K/s, swapping process very fast, even the "agetty"
processes get 64 out of 68 K swapped one or two minutes after booting.
This rate gets lower as nothing but the essential (4K of "agetty", etc) is
left in the physical memory.

> or better use 2.4.6pre3aa1:
>
> 	ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.6pre3aa1.bz2

	I'll test this later as I cannot reboot the machine right now.
Just one question, is this kernel 'stable', I was running 2.4.5-ac9 for
a week without problems, even when 'stress-testing' the emu10k1 and the
mga modules.

	Regards,

	- german

PS: swapoff -a fixes the problem :-)
-------------------------------------------------------------------------
German Gomez Garcia         | "This isn't right.  This isn't even wrong."
<german@piraos.com>         |                         -- Wolfgang Pauli

Re: Strange behaviour of swap under 2.4.5-ac15

[Luigi Genoni]

On Mon, 18 Jun 2001, German Gomez Garcia wrote:

> On Mon, 18 Jun 2001, Andrea Arcangeli wrote:
>
> > On Mon, Jun 18, 2001 at 12:14:01PM +0200, German Gomez Garcia wrote:
> > > 	Hello,
> > >
> > > 	I've running 2.4.5-ac15 for almost a day (22 hours) and I found
> > > some strange behaviour of the kswap, at least it was not present in
> > > 2.4.5-ac9. The swap memory increase with time as the cache dedicated
> > > memory also increase, that is swapping process at a very fast rate, even
> > > when no program is getting more memory. Is that the expected behaviour?
> > > 	An example, with no process running (just the usual daemons and
> > > none of them getting extra memory) the command:
> > >
> > > 	free ; sleep 60; free
> > >
> > >              total       used       free     shared    buffers     cached
> > > Mem:        513416     393184     120232        364      63276     254576
> > > -/+ buffers/cache:      75332     438084
> > > Swap:       530104      14228     515876
> > >
> > >              total       used       free     shared    buffers     cached
> > > Mem:        513416     393192     120224        364      63276     258412
> > > -/+ buffers/cache:      71504     441912
> > > Swap:       530104      18064     512040
> > >
> > > 	Any idea?
> >
> > either apply this patch to 2.4.5ac15:
> >
> > 	ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.5aa3/00_fix-unusable-vm-on-alpha-1
> >
> > (note it is not an alpha specific bug, it's just that I was triggering
> > all the time on alpha so I called the patch that way)
>
> 	It doesn't fix it, swapped memory and cache memory increase at a
> rate of about 40K/s, swapping process very fast, even the "agetty"
> processes get 64 out of 68 K swapped one or two minutes after booting.
> This rate gets lower as nothing but the essential (4K of "agetty", etc) is
> left in the physical memory.
>
I was having similar problems with a sun 3500 with 4 GB of RAM
running 2.4.5 (no problems of this kind with 2.4.4), and this
patch fixed my problems under eavy stress.

I was having the most of memory free and a lot of swap allocated,
and system swapping to dead.

Now it works, with other problems, but when i have time and
permission to try to debug them..... :).

Maybe there could be some HW related reason because of which it fixed for
my sun and not for you...

Luigi

Re: Strange behaviour of swap under 2.4.5-ac15

[Andrea Arcangeli]

On Mon, Jun 18, 2001 at 03:34:43PM +0200, Luigi Genoni wrote:
> Maybe there could be some HW related reason because of which it fixed for

yes, most probably he didn't had any zone empty so the fix couldn't make
a difference for him. German, can you confirm? (if you don't know what
it means just show me a SYSRQ+M and I'll find out myself :)

Since the fix didn't helped it sounds like the memory balancing in -ac
is not working properly for other reasons too.

Andrea

Re: Strange behaviour of swap under 2.4.5-ac15

[Rik van Riel]

On Mon, 18 Jun 2001, Andrea Arcangeli wrote:

> either apply this patch to 2.4.5ac15:
> 
> 	ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.5aa3/00_fix-unusable-vm-on-alpha-1

That one has already been fixed in -pre3 and I think also
in -ac14+ kernels (haven't verified the -ac kernels, though).

The "bug" exists because of a change in refill_inactive(),
which is now a lot closer to being balanced. It's not a bug,
but with the way the statistics are generated it sure look
funny ;)

regards,

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/		http://distro.conectiva.com/

Send all your spam to aardvark@nl.linux.org (spam digging piggy)

Re: Strange behaviour of swap under 2.4.5-ac15

[Andrea Arcangeli]

On Mon, Jun 18, 2001 at 10:43:14AM -0300, Rik van Riel wrote:
> On Mon, 18 Jun 2001, Andrea Arcangeli wrote:
> 
> > either apply this patch to 2.4.5ac15:
> > 
> > 	ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.5aa3/00_fix-unusable-vm-on-alpha-1
> 
> That one has already been fixed in -pre3 and I think also

wrong, I merged it with Linus in pre2 not pre3.

> in -ac14+ kernels (haven't verified the -ac kernels, though).

wrong -ac15 is still missing it.

> which is now a lot closer to being balanced. It's not a bug,

wrong, that was a core showstopper bug and it renders any machine with a
zone empty unusable. (it has nothing to do with beauty or stats)

Andrea

Re: Strange behaviour of swap under 2.4.5-ac15

[Rik van Riel]

On Mon, 18 Jun 2001, Andrea Arcangeli wrote:

> > which is now a lot closer to being balanced. It's not a bug,
>
> wrong, that was a core showstopper bug and it renders any
> machine with a zone empty unusable. (it has nothing to do with
> beauty or stats)

YOUR PATCH fixes a real bug, true.  But that wasn't
what German was complaining about ;)

Rik
--
Executive summary of a recent Microsoft press release:
   "we are concerned about the GNU General Public License (GPL)"


		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: Strange behaviour of swap under 2.4.5-ac15

[Andrea Arcangeli]

On Mon, Jun 18, 2001 at 11:54:09AM -0300, Rik van Riel wrote:
> YOUR PATCH fixes a real bug, true.  But that wasn't
> what German was complaining about ;)

How did you know that before he made his first reply to my email?

Andrea

Re: Strange behaviour of swap under 2.4.5-ac15

[Rik van Riel]

On Mon, 18 Jun 2001, Andrea Arcangeli wrote:
> On Mon, Jun 18, 2001 at 11:54:09AM -0300, Rik van Riel wrote:
> > YOUR PATCH fixes a real bug, true.  But that wasn't
> > what German was complaining about ;)
>
> How did you know that before he made his first reply to my email?

Because he started the thread and I read his first email? ;)

Rik
--
Executive summary of a recent Microsoft press release:
   "we are concerned about the GNU General Public License (GPL)"


		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: Strange behaviour of swap under 2.4.5-ac15

[Andrea Arcangeli]

On Mon, Jun 18, 2001 at 01:44:29PM -0300, Rik van Riel wrote:
> On Mon, 18 Jun 2001, Andrea Arcangeli wrote:
> > On Mon, Jun 18, 2001 at 11:54:09AM -0300, Rik van Riel wrote:
> > > YOUR PATCH fixes a real bug, true.  But that wasn't
> > > what German was complaining about ;)
> >
> > How did you know that before he made his first reply to my email?
> 
> Because he started the thread and I read his first email? ;)

from his first email it sounded like it was such bug, or at least it
wasn't possible to say it wasn't.

Andrea

Re: Strange behaviour of swap under 2.4.5-ac15

[Rik van Riel]

On Mon, 18 Jun 2001, German Gomez Garcia wrote:

> 	I've running 2.4.5-ac15 for almost a day (22 hours) and I found
> some strange behaviour of the kswap, at least it was not present in
> 2.4.5-ac9. The swap memory increase with time as the cache dedicated
> memory also increase, that is swapping process at a very fast rate, even
> when no program is getting more memory. Is that the expected behaviour?

Yes, this is expected behaviour with -ac14, -pre3 and newer.

Note that the memory isn't actually swapped out. It's still
sitting in RAM, being counted as swap cache (and because of
this showing up as "cache" in top).

regards,

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/		http://distro.conectiva.com/

Send all your spam to aardvark@nl.linux.org (spam digging piggy)

Re: Strange behaviour of swap under 2.4.5-ac15

[Andrea Arcangeli]

On Mon, Jun 18, 2001 at 10:37:21AM -0300, Rik van Riel wrote:
> Yes, this is expected behaviour with -ac14, -pre3 and newer.

If that means anything that doesn't happen here based on pre3.

Andrea

Re: Strange behaviour of swap under 2.4.5-ac15

[Mike Galbraith]

On Mon, 18 Jun 2001, Andrea Arcangeli wrote:

> On Mon, Jun 18, 2001 at 10:37:21AM -0300, Rik van Riel wrote:
> > Yes, this is expected behaviour with -ac14, -pre3 and newer.
>
> If that means anything that doesn't happen here based on pre3.

It doesn't happen here either.  Even with (ever so slightly modified)
fair background aging change.

	-Mike

Re: Strange behaviour of swap under 2.4.5-ac15

[German Gomez Garcia]

On Mon, 18 Jun 2001, Andrea Arcangeli wrote:

> On Mon, Jun 18, 2001 at 05:35:54PM +0200, German Gomez Garcia wrote:
> > On Mon, 18 Jun 2001, Andrea Arcangeli wrote:
> >
> > > On Mon, Jun 18, 2001 at 04:41:02PM +0200, German Gomez Garcia wrote:
> > > > so, if there is another way to get that info (maybe some file in /proc?)
> > >
> > > did you compiled with highmem?
> >
> > 	No,
>
> ok then with 512mbyte on an x86 my patch couldn't help. I thought it was
> it because the sympthom was the same (constant kswapd activity and
> constant swap grow) but it obviously is something else. Then try
> 2.4.6pre3aa1 (it is certainly stable).

	You were right, with 2.4.6pre3aa1 it works in the expected way (at
least the way it was expected before :-). I'm now at home so if you need
more info on it, I'll do my best.

	Regards,

	- german

-------------------------------------------------------------------------
German Gomez Garcia         | "This isn't right.  This isn't even wrong."
<german@piraos.com>         |                         -- Wolfgang Pauli