From: Andrew Morton <akpm@linux-foundation.org>
To: Pavel Tatashin <pasha.tatashin@oracle.com>
Cc: linux-mm@kvack.org, sparclinux@vger.kernel.org,
linux-fsdevel@vger.kernel.org, Al Viro <viro@zeniv.linux.org.uk>
Subject: Re: [PATCH v3 4/4] mm: Adaptive hash table scaling
Date: Fri, 3 Mar 2017 15:32:47 -0800 [thread overview]
Message-ID: <20170303153247.f16a31c95404c02a8f3e2c5f@linux-foundation.org> (raw)
In-Reply-To: <1488432825-92126-5-git-send-email-pasha.tatashin@oracle.com>
On Thu, 2 Mar 2017 00:33:45 -0500 Pavel Tatashin <pasha.tatashin@oracle.com> wrote:
> Allow hash tables to scale with memory but at slower pace, when HASH_ADAPT
> is provided every time memory quadruples the sizes of hash tables will only
> double instead of quadrupling as well. This algorithm starts working only
> when memory size reaches a certain point, currently set to 64G.
>
> This is example of dentry hash table size, before and after four various
> memory configurations:
>
> MEMORY SCALE HASH_SIZE
> old new old new
> 8G 13 13 8M 8M
> 16G 13 13 16M 16M
> 32G 13 13 32M 32M
> 64G 13 13 64M 64M
> 128G 13 14 128M 64M
> 256G 13 14 256M 128M
> 512G 13 15 512M 128M
> 1024G 13 15 1024M 256M
> 2048G 13 16 2048M 256M
> 4096G 13 16 4096M 512M
> 8192G 13 17 8192M 512M
> 16384G 13 17 16384M 1024M
> 32768G 13 18 32768M 1024M
> 65536G 13 18 65536M 2048M
OK, but what are the runtime effects? Presumably some workloads will
slow down a bit. How much? How do we know that this is a worthwhile
tradeoff?
If the effect of this change is "undetectable" then those hash tables
are simply too large, and additional tuning is needed, yes?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
WARNING: multiple messages have this Message-ID (diff)
From: Andrew Morton <akpm@linux-foundation.org>
To: Pavel Tatashin <pasha.tatashin@oracle.com>
Cc: linux-mm@kvack.org, sparclinux@vger.kernel.org,
linux-fsdevel@vger.kernel.org, Al Viro <viro@zeniv.linux.org.uk>
Subject: Re: [PATCH v3 4/4] mm: Adaptive hash table scaling
Date: Fri, 03 Mar 2017 23:32:47 +0000 [thread overview]
Message-ID: <20170303153247.f16a31c95404c02a8f3e2c5f@linux-foundation.org> (raw)
In-Reply-To: <1488432825-92126-5-git-send-email-pasha.tatashin@oracle.com>
On Thu, 2 Mar 2017 00:33:45 -0500 Pavel Tatashin <pasha.tatashin@oracle.com> wrote:
> Allow hash tables to scale with memory but at slower pace, when HASH_ADAPT
> is provided every time memory quadruples the sizes of hash tables will only
> double instead of quadrupling as well. This algorithm starts working only
> when memory size reaches a certain point, currently set to 64G.
>
> This is example of dentry hash table size, before and after four various
> memory configurations:
>
> MEMORY SCALE HASH_SIZE
> old new old new
> 8G 13 13 8M 8M
> 16G 13 13 16M 16M
> 32G 13 13 32M 32M
> 64G 13 13 64M 64M
> 128G 13 14 128M 64M
> 256G 13 14 256M 128M
> 512G 13 15 512M 128M
> 1024G 13 15 1024M 256M
> 2048G 13 16 2048M 256M
> 4096G 13 16 4096M 512M
> 8192G 13 17 8192M 512M
> 16384G 13 17 16384M 1024M
> 32768G 13 18 32768M 1024M
> 65536G 13 18 65536M 2048M
OK, but what are the runtime effects? Presumably some workloads will
slow down a bit. How much? How do we know that this is a worthwhile
tradeoff?
If the effect of this change is "undetectable" then those hash tables
are simply too large, and additional tuning is needed, yes?
next prev parent reply other threads:[~2017-03-03 23:32 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-03-02 5:33 [PATCH v3 0/4] Zeroing hash tables in allocator Pavel Tatashin
2017-03-02 5:33 ` Pavel Tatashin
2017-03-02 5:33 ` [PATCH v3 1/4] sparc64: NG4 memset 32 bits overflow Pavel Tatashin
2017-03-02 5:33 ` Pavel Tatashin
2017-03-03 23:34 ` Andrew Morton
2017-03-03 23:34 ` Andrew Morton
2017-03-02 5:33 ` [PATCH v3 2/4] mm: Zeroing hash tables in allocator Pavel Tatashin
2017-03-02 5:33 ` Pavel Tatashin
2017-03-02 5:33 ` [PATCH v3 3/4] mm: Updated callers to use HASH_ZERO flag Pavel Tatashin
2017-03-02 5:33 ` Pavel Tatashin
2017-03-02 5:33 ` [PATCH v3 4/4] mm: Adaptive hash table scaling Pavel Tatashin
2017-03-02 5:33 ` Pavel Tatashin
2017-03-03 23:32 ` Andrew Morton [this message]
2017-03-03 23:32 ` Andrew Morton
2017-04-26 20:11 ` Michal Hocko
2017-04-26 20:11 ` Michal Hocko
2017-05-02 8:04 ` Michal Hocko
2017-05-02 8:04 ` Michal Hocko
2017-05-04 18:23 ` Pasha Tatashin
2017-05-04 18:23 ` Pasha Tatashin
2017-05-04 18:28 ` Pasha Tatashin
2017-05-04 18:28 ` Pasha Tatashin
2017-05-05 13:30 ` Michal Hocko
2017-05-05 13:30 ` Michal Hocko
2017-05-05 15:33 ` Pasha Tatashin
2017-05-05 15:33 ` Pasha Tatashin
2017-05-09 9:46 ` Michal Hocko
2017-05-09 9:46 ` Michal Hocko
2017-05-09 9:46 ` Michal Hocko
2017-05-09 13:07 ` Pasha Tatashin
2017-05-09 13:07 ` Pasha Tatashin
2017-05-05 13:29 ` Michal Hocko
2017-05-05 13:29 ` Michal Hocko
2017-05-17 15:51 ` Pasha Tatashin
2017-05-17 15:51 ` Pasha Tatashin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170303153247.f16a31c95404c02a8f3e2c5f@linux-foundation.org \
--to=akpm@linux-foundation.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=pasha.tatashin@oracle.com \
--cc=sparclinux@vger.kernel.org \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.