2.4.20-18 size-4096 memory leaks

2.4.20-18 size-4096 memory leaks

[yuval yeret]

Hi,

I'm seeing a constant leak in size-4096 on a machine running 2.4.20-18 SMP 
BIGMEM, which might / might not be related to the machine finally going out 
of memory and going into a hang.

I saw a discussion around similar problems in 2.6.0 (2.6.0-test5/6 (and 
probably 7 too) size-4096 memory leak - http://lkml.org/lkml/2003/10/17/5 )
and an ext3 patch was suggested by Andrew Morton.

>From a brief look the code in 2.4 it seems like the patch might be relevant 
here as well. Is the size-4096 leak a known issue for 2.4 ?
Is the 2.6 patch applicable in 2.4 as well ?


Thanks,

--
Yuval Yeret
Yuval at exanet dot com
Exanet
http://www.exanet.com

_________________________________________________________________
The new MSN 8: smart spam protection and 2 months FREE*  
http://join.msn.com/?page=features/junkmail

Re: 2.4.20-18 size-4096 memory leaks

[Henrik Storner]

In <BAY2-F65DwLK4adtttY00010f98@hotmail.com> "yuval yeret" <yuval_yeret@hotmail.com> writes:

>I'm seeing a constant leak in size-4096 on a machine running 2.4.20-18 SMP 
>BIGMEM, which might / might not be related to the machine finally going out 
>of memory and going into a hang.

This sounds like one of the Red Hat kernels - those are immensely
patched and very different from the original Linux kernels normally
discussed here.

However ...

>I saw a discussion around similar problems in 2.6.0 (2.6.0-test5/6 (and 
>probably 7 too) size-4096 memory leak - http://lkml.org/lkml/2003/10/17/5 )
>and an ext3 patch was suggested by Andrew Morton.

>From a brief look the code in 2.4 it seems like the patch might be relevant 
>here as well. Is the size-4096 leak a known issue for 2.4 ?
>Is the 2.6 patch applicable in 2.4 as well ?

There definitely is a memory leak in the Red Hat 9 kernels, including
the 2.4.20-20.9smp kernel (seems to be in the uniprocessor build as well).
It leaks mm_struct slabs - see the trend graphs I do from the slabinfo
data, available at 

http://tyge.sslug.dk/bb-cgi/larrd-grapher.cgi?host=tyge.sslug.dk&service=slabinfo

The blue curve is the mm_struct slab allocation (column 5, the "active
pages" column from /proc/slabinfo) - the way it continually grows from
the system is rebooted IMHO points to a memory leak somewhere. This is
very clear from the third graph, which goes back the last 48 days,
during which we had two reboots of the system (hardware related).

The standard kernels from www.kernel.org do not have this problem,
so I am fairly certain it is in one of the Red Hat patches.

Others also see it:
 https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=91090


Henrik
-- 
Henrik Storner <henrik@hswn.dk> 

Re: 2.4.20-18 size-4096 memory leaks

[Stephen C. Tweedie]

Hi,

On Wed, 2003-11-26 at 09:29, yuval yeret wrote:

> I saw a discussion around similar problems in 2.6.0 (2.6.0-test5/6 (and 
> probably 7 too) size-4096 memory leak - http://lkml.org/lkml/2003/10/17/5 )
> and an ext3 patch was suggested by Andrew Morton.
> 
> From a brief look the code in 2.4 it seems like the patch might be relevant 
> here as well. Is the size-4096 leak a known issue for 2.4 ?
> Is the 2.6 patch applicable in 2.4 as well ?

No.  The journal_release_buffer() code is not used, or even enabled, on
2.4.  There is one set of patches which can use it on 2.4 --- the EA/ACL
code does, but only for extended attributes, and the leak mentioned
above only affects release_buffer() on bitmap buffers.

Cheers,
 Stephen

Re: 2.4.20-18 size-4096 memory leaks

[yuval yeret]

>From: "Stephen C. Tweedie" <sct@redhat.com>
>
>That's not a leak, it simply sounds like cache effects.  atime updates
>result in journal commits under ext3, and those use at least a couple of
>buffers at a time (one for the metadata descriptor block in the journal,
>one for the journal commit.)  Those aren't leaks --- they are temporary
>use of cache, and once the IO has complete the memory can be immediately
>reclaimed by the kernel if it is needed for anything else.

I'm using noatime for my ext3 mount points ...

>
>Cheers,
>  Stephen
>
>

_________________________________________________________________
Help STOP SPAM with the new MSN 8 and get 2 months FREE*  
http://join.msn.com/?page=features/junkmail

Re: 2.4.20-18 size-4096 memory leaks

[Stephen C. Tweedie]

Hi,

On Wed, 2003-11-26 at 10:57, Jean Delvare wrote:

> I just wanted to let you know that I have been experiencing similar
> leaks. So far, I wasn't enable to find where the leak was, but your
> theory matches my observations:
> 
> 1* On two systems running 2.4.20-2.4.22 kernels, I observed that the
> free memory as reported by top was going down regularly, by blocks of 4
> or 8kB at an average rate of 90kB/min. Sometimes the value would
> stabilize, but I couldn't understand why. What was lost as "free"
> memory
> increased "buffers" from the same amount.

That's not a leak, it simply sounds like cache effects.  atime updates
result in journal commits under ext3, and those use at least a couple of
buffers at a time (one for the metadata descriptor block in the journal,
one for the journal commit.)  Those aren't leaks --- they are temporary
use of cache, and once the IO has complete the memory can be immediately
reclaimed by the kernel if it is needed for anything else.

Cheers, 
 Stephen

Re: 2.4.20-18 size-4096 memory leaks

[Jean Delvare]

Hi Yuval, hi list,

> I'm seeing a constant leak in size-4096 on a machine running
> 2.4.20-18 SMP BIGMEM, which might / might not be related to the
> machine finally going out of memory and going into a hang.

I just wanted to let you know that I have been experiencing similar
leaks. So far, I wasn't enable to find where the leak was, but your
theory matches my observations:

1* On two systems running 2.4.20-2.4.22 kernels, I observed that the
free memory as reported by top was going down regularly, by blocks of 4
or 8kB at an average rate of 90kB/min. Sometimes the value would
stabilize, but I couldn't understand why. What was lost as "free"
memory
increased "buffers" from the same amount. Still that value doesn't
exceed a few dozen MB and sometimes goes does by large blocks, so I was
wondering if there was a real leak or if it was just some kind of
regular prebuffering.

2* On two other, seemingly similar systems, the memory leak wouldn't
occur. It happens that these two systems do *not* use ext3, while the
fisrt two *do* use ext3.

3* The leak isn't distribution specific. I experienced it on both a
Mandrake-shipped kernel and a self-compiled one on a Slackware system.

Your theory that it comes from the ext3 driver makes full sense. I'll
confirm that later today, by using ext2 only on one of the leaking
systems (without changing kernels of course).

The other leaking system is used as a development server. It used to
become very slow after several days. We are now restarting it every
monday and have no significant slowdown anymore. Having other things to
do, I stopped my experiments there, but could do some more now if it
can help.

-- 
Jean Delvare
http://www.ensicaen.ismra.fr/~delvare/

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

2.4.20-18 size-4096 memory leaks

[yuval yeret]

(hopefully plain text this time)

Hi,
I'm seeing a constant leak in size-4096 on a machine running 2.4.20-18 SMP 
BIGMEM, which might / might not be related to the machine finally going out 
of memory and going into a hang.
I saw a discussion around similar problems in 2.6.0 (2.6.0-test5/6 (and 
probably 7 too) size-4096 memory leak - http://lkml.org/lkml/2003/10/17/5 )
and an ext3 patch was suggested by Andrew Morton.
>From a brief look the code in 2.4 it seems like the patch might be relevant 
here as well. Is the size-4096 leak a known issue for 2.4 ?
Is the 2.6 patch applicable in 2.4 as well ?
Thanks,
--
Yuval Yeret
Yuval at exanet dot com
Exanet
http://www.exanet.com

_________________________________________________________________
The new MSN 8: advanced junk mail protection and 2 months FREE* 
http://join.msn.com/?page=features/junkmail