All of lore.kernel.org
 help / color / mirror / Atom feed
From: Rafael Aquini <aquini@linux.com>
To: Russ Anderson <rja@sgi.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Andrea Arcangeli <aarcange@redhat.com>,
	linux-mm <linux-mm@kvack.org>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	Christoph Lameter <cl@linux.com>,
	rja@americas.sgi.com
Subject: [PATCH] mm: fix negative commitlimit when gigantic hugepages are allocated
Date: Thu, 2 Jun 2011 23:55:57 -0300	[thread overview]
Message-ID: <20110603025555.GA10530@optiplex.tchesoft.com> (raw)
In-Reply-To: <20110602040821.GA7934@sgi.com>

When 1GB hugepages are allocated on a system, free(1) reports
less available memory than what really is installed in the box.
Also, if the total size of hugepages allocated on a system is
over half of the total memory size, CommitLimit becomes
a negative number.

The problem is that gigantic hugepages (order > MAX_ORDER)
can only be allocated at boot with bootmem, thus its frames
are not accounted to 'totalram_pages'. However,  they are
accounted to hugetlb_total_pages()

What happens to turn CommitLimit into a negative number
is this calculation, in fs/proc/meminfo.c:

        allowed = ((totalram_pages - hugetlb_total_pages())
                * sysctl_overcommit_ratio / 100) + total_swap_pages;

A similar calculation occurs in __vm_enough_memory() in mm/mmap.c.

Also, every vm statistic which depends on 'totalram_pages' will render
confusing values, as if system were 'missing' some part of its memory.

Reported-by: Russ Anderson <rja@sgi.com>
Signed-off-by: Rafael Aquini <aquini@linux.com>
---
 mm/hugetlb.c |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index f33bb31..c67dd0f 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -1111,6 +1111,14 @@ static void __init gather_bootmem_prealloc(void)
 		WARN_ON(page_count(page) != 1);
 		prep_compound_huge_page(page, h->order);
 		prep_new_huge_page(h, page, page_to_nid(page));
+
+		/* if we had gigantic hugepages allocated at boot time,
+		 * we need to reinstate the 'stolen' pages to totalram_pages,
+		 * in order to fix confusing memory reports from free(1)
+		 * and another side-effects, like CommitLimit going negative.
+		 */
+		if (h->order > (MAX_ORDER - 1))
+			totalram_pages += 1 << h->order;
 	}
 }
 
-- 
1.7.4.4


WARNING: multiple messages have this Message-ID (diff)
From: Rafael Aquini <aquini@linux.com>
To: Russ Anderson <rja@sgi.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Andrea Arcangeli <aarcange@redhat.com>,
	linux-mm <linux-mm@kvack.org>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	Christoph Lameter <cl@linux.com>,
	rja@americas.sgi.com
Subject: [PATCH] mm: fix negative commitlimit when gigantic hugepages are allocated
Date: Thu, 2 Jun 2011 23:55:57 -0300	[thread overview]
Message-ID: <20110603025555.GA10530@optiplex.tchesoft.com> (raw)
In-Reply-To: <20110602040821.GA7934@sgi.com>

When 1GB hugepages are allocated on a system, free(1) reports
less available memory than what really is installed in the box.
Also, if the total size of hugepages allocated on a system is
over half of the total memory size, CommitLimit becomes
a negative number.

The problem is that gigantic hugepages (order > MAX_ORDER)
can only be allocated at boot with bootmem, thus its frames
are not accounted to 'totalram_pages'. However,  they are
accounted to hugetlb_total_pages()

What happens to turn CommitLimit into a negative number
is this calculation, in fs/proc/meminfo.c:

        allowed = ((totalram_pages - hugetlb_total_pages())
                * sysctl_overcommit_ratio / 100) + total_swap_pages;

A similar calculation occurs in __vm_enough_memory() in mm/mmap.c.

Also, every vm statistic which depends on 'totalram_pages' will render
confusing values, as if system were 'missing' some part of its memory.

Reported-by: Russ Anderson <rja@sgi.com>
Signed-off-by: Rafael Aquini <aquini@linux.com>
---
 mm/hugetlb.c |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index f33bb31..c67dd0f 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -1111,6 +1111,14 @@ static void __init gather_bootmem_prealloc(void)
 		WARN_ON(page_count(page) != 1);
 		prep_compound_huge_page(page, h->order);
 		prep_new_huge_page(h, page, page_to_nid(page));
+
+		/* if we had gigantic hugepages allocated at boot time,
+		 * we need to reinstate the 'stolen' pages to totalram_pages,
+		 * in order to fix confusing memory reports from free(1)
+		 * and another side-effects, like CommitLimit going negative.
+		 */
+		if (h->order > (MAX_ORDER - 1))
+			totalram_pages += 1 << h->order;
 	}
 }
 
-- 
1.7.4.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2011-06-03  2:57 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-05-18 15:34 [PATCH] [BUGFIX] mm: hugepages can cause negative commitlimit Russ Anderson
2011-05-18 15:34 ` Russ Anderson
2011-05-19  0:51 ` Rafael Aquini
2011-05-19  4:56   ` Russ Anderson
2011-05-19  4:56     ` Russ Anderson
2011-05-19 13:37     ` Rafael Aquini
2011-05-19 22:11       ` Russ Anderson
2011-05-19 22:11         ` Russ Anderson
2011-05-20 20:04         ` Andrew Morton
2011-05-20 20:04           ` Andrew Morton
2011-05-20 22:30           ` Rafael Aquini
2011-05-20 22:30             ` Rafael Aquini
2011-05-26 21:07             ` Rafael Aquini
2011-05-26 21:07               ` Rafael Aquini
2011-05-27 22:22               ` Russ Anderson
2011-05-27 22:22                 ` Russ Anderson
2011-06-02  4:08               ` Russ Anderson
2011-06-02  4:08                 ` Russ Anderson
2011-06-03  2:55                 ` Rafael Aquini [this message]
2011-06-03  2:55                   ` [PATCH] mm: fix negative commitlimit when gigantic hugepages are allocated Rafael Aquini
2011-06-03 12:07                   ` Russ Anderson
2011-06-03 12:07                     ` Russ Anderson
2011-06-09 23:44                   ` Andrew Morton
2011-06-09 23:44                     ` Andrew Morton
2011-06-13 21:11                     ` Rafael Aquini
2011-06-13 21:11                       ` Rafael Aquini
2011-06-13 21:31                       ` Andrew Morton
2011-06-13 21:31                         ` Andrew Morton
2011-06-03  3:08                 ` [PATCH] [BUGFIX] mm: hugepages can cause negative commitlimit Rafael Aquini
2011-06-03  3:08                   ` Rafael Aquini

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20110603025555.GA10530@optiplex.tchesoft.com \
    --to=aquini@linux.com \
    --cc=aarcange@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=cl@linux.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=rja@americas.sgi.com \
    --cc=rja@sgi.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.