From: Russ Anderson <rja@sgi.com> To: Rafael Aquini <aquini@linux.com> Cc: Andrew Morton <akpm@linux-foundation.org>, Andrea Arcangeli <aarcange@redhat.com>, linux-mm <linux-mm@kvack.org>, linux-kernel <linux-kernel@vger.kernel.org>, Christoph Lameter <cl@linux.com>, rja@americas.sgi.com Subject: Re: [PATCH] [BUGFIX] mm: hugepages can cause negative commitlimit Date: Fri, 27 May 2011 17:22:25 -0500 [thread overview] Message-ID: <20110527222225.GA8561@sgi.com> (raw) In-Reply-To: <20110526210751.GA14819@optiplex.tchesoft.com> On Thu, May 26, 2011 at 06:07:53PM -0300, Rafael Aquini wrote: > On Fri, May 20, 2011 at 07:30:32PM -0300, Rafael Aquini wrote: > > On Fri, May 20, 2011 at 01:04:11PM -0700, Andrew Morton wrote: > > > On Thu, 19 May 2011 17:11:01 -0500 > > > Russ Anderson <rja@sgi.com> wrote: > > > > > > > OK, I see your point. The root problem is hugepages allocated at boot are > > > > subtracted from totalram_pages but hugepages allocated at run time are not. > > > > Correct me if I've mistate it or are other conditions. > > > > > > > > By "allocated at run time" I mean "echo 1 > /proc/sys/vm/nr_hugepages". > > > > That allocation will not change totalram_pages but will change > > > > hugetlb_total_pages(). > > > > > > > > How best to fix this inconsistency? Should totalram_pages include or exclude > > > > hugepages? What are the implications? > > > > > > The problem is that hugetlb_total_pages() is trying to account for two > > > different things, while totalram_pages accounts for only one of those > > > things, yes? > > > > > > One fix would be to stop accounting for huge pages in totalram_pages > > > altogether. That might break other things so careful checking would be > > > needed. > > > > > > Or we stop accounting for the boot-time allocated huge pages in > > > hugetlb_total_pages(). Split the two things apart altogether and > > > account for boot-time allocated and runtime-allocated pages separately. This > > > souds saner to me - it reflects what's actually happening in the kernel. > > > > Perhaps we can just reinstate the # of pages "stealed" at early boot allocation > > later, when hugetlb_init() calls gather_bootmem_prealloc() > > > > diff --git a/mm/hugetlb.c b/mm/hugetlb.c > > index 8ee3bd8..d606c9c 100644 > > --- a/mm/hugetlb.c > > +++ b/mm/hugetlb.c > > @@ -1111,6 +1111,7 @@ static void __init gather_bootmem_prealloc(void) > > WARN_ON(page_count(page) != 1); > > prep_compound_huge_page(page, h->order); > > prep_new_huge_page(h, page, page_to_nid(page)); > > + totalram_pages += 1 << h->order; > > } > > } > > Howdy Russ, > > Were you able to confirm if that proposed change fix the issue you've reported? Sorry, I have been distracted. I will get to it shortly. > Although I've tested it with usual size hugepages and it did not messed things up, > I'm not able to test it with GB hugepages, as I do not have any proc with "pdpe1gb" flag available. > > Thanks in advance! > Cheers! > -- > Rafael Aquini <aquini@linux.com> -- Russ Anderson, OS RAS/Partitioning Project Lead SGI - Silicon Graphics Inc rja@sgi.com
WARNING: multiple messages have this Message-ID (diff)
From: Russ Anderson <rja@sgi.com> To: Rafael Aquini <aquini@linux.com> Cc: Andrew Morton <akpm@linux-foundation.org>, Andrea Arcangeli <aarcange@redhat.com>, linux-mm <linux-mm@kvack.org>, linux-kernel <linux-kernel@vger.kernel.org>, Christoph Lameter <cl@linux.com>, rja@americas.sgi.com Subject: Re: [PATCH] [BUGFIX] mm: hugepages can cause negative commitlimit Date: Fri, 27 May 2011 17:22:25 -0500 [thread overview] Message-ID: <20110527222225.GA8561@sgi.com> (raw) In-Reply-To: <20110526210751.GA14819@optiplex.tchesoft.com> On Thu, May 26, 2011 at 06:07:53PM -0300, Rafael Aquini wrote: > On Fri, May 20, 2011 at 07:30:32PM -0300, Rafael Aquini wrote: > > On Fri, May 20, 2011 at 01:04:11PM -0700, Andrew Morton wrote: > > > On Thu, 19 May 2011 17:11:01 -0500 > > > Russ Anderson <rja@sgi.com> wrote: > > > > > > > OK, I see your point. The root problem is hugepages allocated at boot are > > > > subtracted from totalram_pages but hugepages allocated at run time are not. > > > > Correct me if I've mistate it or are other conditions. > > > > > > > > By "allocated at run time" I mean "echo 1 > /proc/sys/vm/nr_hugepages". > > > > That allocation will not change totalram_pages but will change > > > > hugetlb_total_pages(). > > > > > > > > How best to fix this inconsistency? Should totalram_pages include or exclude > > > > hugepages? What are the implications? > > > > > > The problem is that hugetlb_total_pages() is trying to account for two > > > different things, while totalram_pages accounts for only one of those > > > things, yes? > > > > > > One fix would be to stop accounting for huge pages in totalram_pages > > > altogether. That might break other things so careful checking would be > > > needed. > > > > > > Or we stop accounting for the boot-time allocated huge pages in > > > hugetlb_total_pages(). Split the two things apart altogether and > > > account for boot-time allocated and runtime-allocated pages separately. This > > > souds saner to me - it reflects what's actually happening in the kernel. > > > > Perhaps we can just reinstate the # of pages "stealed" at early boot allocation > > later, when hugetlb_init() calls gather_bootmem_prealloc() > > > > diff --git a/mm/hugetlb.c b/mm/hugetlb.c > > index 8ee3bd8..d606c9c 100644 > > --- a/mm/hugetlb.c > > +++ b/mm/hugetlb.c > > @@ -1111,6 +1111,7 @@ static void __init gather_bootmem_prealloc(void) > > WARN_ON(page_count(page) != 1); > > prep_compound_huge_page(page, h->order); > > prep_new_huge_page(h, page, page_to_nid(page)); > > + totalram_pages += 1 << h->order; > > } > > } > > Howdy Russ, > > Were you able to confirm if that proposed change fix the issue you've reported? Sorry, I have been distracted. I will get to it shortly. > Although I've tested it with usual size hugepages and it did not messed things up, > I'm not able to test it with GB hugepages, as I do not have any proc with "pdpe1gb" flag available. > > Thanks in advance! > Cheers! > -- > Rafael Aquini <aquini@linux.com> -- Russ Anderson, OS RAS/Partitioning Project Lead SGI - Silicon Graphics Inc rja@sgi.com -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2011-05-27 22:22 UTC|newest] Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top 2011-05-18 15:34 [PATCH] [BUGFIX] mm: hugepages can cause negative commitlimit Russ Anderson 2011-05-18 15:34 ` Russ Anderson 2011-05-19 0:51 ` Rafael Aquini 2011-05-19 4:56 ` Russ Anderson 2011-05-19 4:56 ` Russ Anderson 2011-05-19 13:37 ` Rafael Aquini 2011-05-19 22:11 ` Russ Anderson 2011-05-19 22:11 ` Russ Anderson 2011-05-20 20:04 ` Andrew Morton 2011-05-20 20:04 ` Andrew Morton 2011-05-20 22:30 ` Rafael Aquini 2011-05-20 22:30 ` Rafael Aquini 2011-05-26 21:07 ` Rafael Aquini 2011-05-26 21:07 ` Rafael Aquini 2011-05-27 22:22 ` Russ Anderson [this message] 2011-05-27 22:22 ` Russ Anderson 2011-06-02 4:08 ` Russ Anderson 2011-06-02 4:08 ` Russ Anderson 2011-06-03 2:55 ` [PATCH] mm: fix negative commitlimit when gigantic hugepages are allocated Rafael Aquini 2011-06-03 2:55 ` Rafael Aquini 2011-06-03 12:07 ` Russ Anderson 2011-06-03 12:07 ` Russ Anderson 2011-06-09 23:44 ` Andrew Morton 2011-06-09 23:44 ` Andrew Morton 2011-06-13 21:11 ` Rafael Aquini 2011-06-13 21:11 ` Rafael Aquini 2011-06-13 21:31 ` Andrew Morton 2011-06-13 21:31 ` Andrew Morton 2011-06-03 3:08 ` [PATCH] [BUGFIX] mm: hugepages can cause negative commitlimit Rafael Aquini 2011-06-03 3:08 ` Rafael Aquini
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20110527222225.GA8561@sgi.com \ --to=rja@sgi.com \ --cc=aarcange@redhat.com \ --cc=akpm@linux-foundation.org \ --cc=aquini@linux.com \ --cc=cl@linux.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=rja@americas.sgi.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.