linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Mike Kravetz <mike.kravetz@oracle.com>
To: Matthew Wilcox <willy@infradead.org>
Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org
Subject: Re: THP backed thread stacks
Date: Fri, 17 Mar 2023 11:46:32 -0700	[thread overview]
Message-ID: <20230317184632.GA69459@monkey> (raw)
In-Reply-To: <ZBSo+mLUOsKvy3rC@casper.infradead.org>

On 03/17/23 17:52, Matthew Wilcox wrote:
> On Mon, Mar 06, 2023 at 03:57:30PM -0800, Mike Kravetz wrote:
> > One of our product teams recently experienced 'memory bloat' in their
> > environment.  The application in this environment is the JVM which
> > creates hundreds of threads.  Threads are ultimately created via
> > pthread_create which also creates the thread stacks.  pthread attributes
> > are modified so that stacks are 2MB in size.  It just so happens that
> > due to allocation patterns, all their stacks are at 2MB boundaries.  The
> > system has THP always set, so a huge page is allocated at the first
> > (write) fault when libpthread initializes the stack.
> 
> Do you happen to have an strace (or similar) so we can understand what
> the application is doing?
> 
> My understanding is that for a normal app (like, say, 'cat'), we'll
> allow up to an 8MB stack, but we only create a VMA that is 4kB in size
> and set the VM_GROWSDOWN flag on it (to allow it to magically grow).
> Therefore we won't create a 2MB page because the VMA is too small.
> 
> It sounds like the pthread library is maybe creating a 2MB stack as
> a 2MB VMA, and that's why we're seeing this behaviour?

Yes, pthread stacks create a VMA equal to stack size which is different
than 'main thread' stack.  The 2MB size for pthread stacks created by
JVM is actually them explicitly requesting the size (8MB default).

We have a good understanding of what is happening.  Behavior actually
changed a bit with glibc versions in OL7 vs OL8.  Do note that THP usage
is somewhat out of the control of an application IF they rely on
glibc/pthread to allocate stacks.  Only way for application to make sure
pthread stacks do not use THP would be for them to allocate themselves.
Then, they would need to set up the guard page themselves.  They would
also need to monitor the status of all threads to determine when stacks
could be deleted.  A bunch of extra code that glibc/pthread already does
for free.

Oracle glibc team is also involved, and it 'looks' like they may have
upstream buy in to add a flag to explicitly enable or disable hugepages
on pthread stacks.

It seems like concensus from mm community is that we should not
treat stacks any differently than any other mappings WRT THP.  That is
OK, just wanted to throw it out there.
-- 
Mike Kravetz

  reply	other threads:[~2023-03-17 18:46 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-03-06 23:57 THP backed thread stacks Mike Kravetz
2023-03-07  0:15 ` Peter Xu
2023-03-07  0:40   ` Mike Kravetz
2023-03-08 19:02     ` Mike Kravetz
2023-03-09 22:38       ` Zach O'Keefe
2023-03-09 23:33         ` Mike Kravetz
2023-03-10  0:05           ` Zach O'Keefe
2023-03-10  1:40             ` William Kucharski
2023-03-10 11:25               ` David Hildenbrand
2023-03-11 12:24                 ` William Kucharski
     [not found]                 ` <20230312005549.2609-1-hdanton@sina.com>
2023-03-12  4:39                   ` William Kucharski
2023-03-10 22:02             ` Yang Shi
2023-03-07 10:10 ` David Hildenbrand
2023-03-07 19:02   ` Mike Kravetz
2023-03-07 13:36 ` Mike Rapoport
2023-03-17 17:52 ` Matthew Wilcox
2023-03-17 18:46   ` Mike Kravetz [this message]
2023-03-20 11:12     ` David Hildenbrand
2023-03-20 17:46       ` William Kucharski
2023-03-20 17:52         ` David Hildenbrand
2023-03-20 18:06         ` Mike Kravetz
2023-03-18 12:58   ` David Laight

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230317184632.GA69459@monkey \
    --to=mike.kravetz@oracle.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=willy@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).