All of lore.kernel.org
 help / color / mirror / Atom feed
From: Christoph Lameter <cl@linux.com>
To: Peter Kruse <pk@q-leap.com>
Cc: linux-kernel@vger.kernel.org
Subject: Re: I have a blaze of 353 page allocation failures, all alike
Date: Tue, 15 Feb 2011 11:30:02 -0600 (CST)	[thread overview]
Message-ID: <alpine.DEB.2.00.1102151040560.10511@router.home> (raw)
In-Reply-To: <4D5A2EDB.8060603@q-leap.com>

On Tue, 15 Feb 2011, Peter Kruse wrote:

> > > we have set vm.min_free_kbytes = 2097152 but the problem
> > > obviously did not go away.
> >
> > 2GB of reserves? How much memory does your system have?
>
> 48GB

Ok then you just may potentially clog up the DMA zones. Maybe set the
reserves to a reasonable level like 10M or so?

How many buffers are configured at the various levels for the device that
is receiving messages? I guess that may be a bit on the high side?

> > Could you post the entire messages from the kernel log? We need the OOM
> > info to figure out more about the problem.
> >
>
> I attach one of the call traces, or would it be better if I send the
> kern.log (about 6MB)?

The call traces are sufficient but the traces vanished when I hit reply.
Include them inline next time. It would be good to have the log starting
at the last system boot. There is some information cut off that I would to
see.

An atomic order 1 allocation failed and led to the OOM but it seems that
there is still ample memory available. Slab is in "fallback_alloc" so
something went wrong with the regular allocation attempt. Any use of
cpusets or cgroups?

A significant amount of memory has been allocated to reclaimable slabs.
I guess these are the socket buffers?

Feb 10 11:59:49 beosrv1-t kernel: [1968911.211777] Node 0 Normal
free:965164kB min:917952kB low:1147440kB high:1376928kB
active_anon:2742680kB inactive_anon:293184kB active_file:4801512kB
inactive_file:11129708kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:21719040kB mlocked:0kB dirty:600kB
writeback:0kB mapped:26356kB shmem:4896kB slab_reclaimable:1780208kB <-----!!
slab_unreclaimable:199576kB kernel_stack:1576kB pagetables:22956kB
unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0
all_unreclaimable? no

Could you try to reduce the number of network buffers?


  reply	other threads:[~2011-02-15 17:30 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-02-10 15:03 I have a blaze of 353 page allocation failures, all alike Peter Kruse
2011-02-14 16:49 ` Christoph Lameter
2011-02-15  7:44   ` Peter Kruse
2011-02-15 17:30     ` Christoph Lameter [this message]
2011-02-16 12:22       ` Peter Kruse
2011-02-16 15:59         ` Christoph Lameter
2011-02-16 16:03           ` Peter Kruse
2011-02-16 16:14             ` Christoph Lameter
2011-02-17  7:31               ` Peter Kruse
2011-02-17 17:03                 ` Christoph Lameter
2011-02-18 12:30                   ` Peter Kruse
2011-02-24 12:01                     ` Peter Kruse
2011-04-12 15:01                       ` Peter Kruse
2011-04-12 18:08                         ` Christoph Lameter
2011-04-13  1:34                           ` David Rientjes
2011-04-13  7:13                             ` Peter Kruse
2011-04-13 16:17                               ` Christoph Lameter
2011-05-19 11:56                                 ` Peter Kruse
2011-05-19 16:00                                   ` Christoph Lameter
2011-05-23  6:34                                     ` Peter Kruse
     [not found]                                 ` <4E09BEA1.1080501@q-leap.de>
     [not found]                                   ` <alpine.DEB.2.00.1107051013500.16869@router.home>
2011-07-05 17:20                                     ` Mel Gorman
2011-07-06  4:16                                       ` Dave Chinner
2011-07-06  6:50                                         ` Peter Kruse
2011-07-06 14:31                                           ` Christoph Lameter
2011-07-06 15:15                                             ` Peter Kruse
2011-07-06 15:30                                               ` Christoph Lameter
2011-11-24 10:53                                                 ` Peter Kruse

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=alpine.DEB.2.00.1102151040560.10511@router.home \
    --to=cl@linux.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=pk@q-leap.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.