From: Joonsoo Kim <iamjoonsoo.kim@lge.com>
To: Vladimir Davydov <vdavydov@parallels.com>
Cc: akpm@linux-foundation.org, cl@linux.com, rientjes@google.com,
penberg@kernel.org, hannes@cmpxchg.org, mhocko@suse.cz,
linux-kernel@vger.kernel.org, linux-mm@kvack.org
Subject: Re: [PATCH -mm v3 8/8] slab: do not keep free objects/slabs on dead memcg caches
Date: Fri, 27 Jun 2014 15:05:34 +0900 [thread overview]
Message-ID: <20140627060534.GC9511@js1304-P5Q-DELUXE> (raw)
In-Reply-To: <20140625134545.GB22340@esperanza>
On Wed, Jun 25, 2014 at 05:45:45PM +0400, Vladimir Davydov wrote:
> On Tue, Jun 24, 2014 at 04:38:41PM +0900, Joonsoo Kim wrote:
> > On Fri, Jun 13, 2014 at 12:38:22AM +0400, Vladimir Davydov wrote:
> > And, you said that this way of implementation would be slow because
> > there could be many object in dead caches and this implementation
> > needs node spin_lock on each object freeing. Is it no problem now?
> >
> > If you have any performance data about this implementation and
> > alternative one, could you share it?
>
> I ran some tests on a 2 CPU x 6 core x 2 HT box. The kernel was compiled
> with a config taken from a popular distro, so it had most of debug
> options turned off.
>
> ---
>
> TEST #1: Each logical CPU executes a task that frees 1M objects
> allocated from the same cache. All frees are node-local.
>
> RESULTS:
>
> objsize (bytes) | cache is dead? | objects free time (ms)
> ----------------+----------------+-----------------------
> 64 | - | 373 +- 5
> - | + | 1300 +- 6
> | |
> 128 | - | 387 +- 6
> - | + | 1337 +- 6
> | |
> 256 | - | 484 +- 4
> - | + | 1407 +- 6
> | |
> 512 | - | 686 +- 5
> - | + | 1561 +- 18
> | |
> 1024 | - | 1073 +- 11
> - | + | 1897 +- 12
>
> TEST #2: Each logical CPU executes a task that removes 1M empty files
> from its own RAMFS mount. All frees are node-local.
>
> RESULTS:
>
> cache is dead? | files removal time (s)
> ----------------+----------------------------------
> - | 15.57 +- 0.55 (base)
> + | 16.80 +- 0.62 (base + 8%)
>
> ---
>
> So, according to TEST #1 the relative slowdown introduced by zapping per
> cpu arrays is really dreadful - it can be up to 4x! However, the
> absolute numbers aren't that huge - ~1 second for 24 million objects.
> If we do something else except kfree the slowdown shouldn't be that
> visible IMO.
>
> TEST #2 is an attempt to estimate how zapping of per cpu arrays will
> affect FS objects destruction, which is the most common case of dead
> caches usage. To avoid disk-bound operations it uses RAMFS. From the
> test results it follows that the relative slowdown of massive file
> deletion is within 2 stdev, which looks decent.
>
> Anyway, the alternative approach (reaping dead caches periodically)
> won't have this kfree slowdown at all. However, periodic reaping can
> become a real disaster as the system evolves and the number of dead
> caches grows. Currently I don't know how we can estimate real life
> effects of this. If you have any ideas, please let me know.
>
Hello,
I have no idea here. I don't have much experience on large scale
system. But, current implementation would also have big trouble if
system is larger than yours.
I think that Christoph can say something about this result.
Christoph,
Is it tolerable result for large scale system? Or do we need to find
another solution?
Thanks.
next prev parent reply other threads:[~2014-06-27 6:00 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-06-12 20:38 [PATCH -mm v3 0/8] memcg/slab: reintroduce dead cache self-destruction Vladimir Davydov
2014-06-12 20:38 ` [PATCH -mm v3 1/8] memcg: cleanup memcg_cache_params refcnt usage Vladimir Davydov
2014-06-12 20:38 ` [PATCH -mm v3 2/8] memcg: destroy kmem caches when last slab is freed Vladimir Davydov
2014-06-12 20:38 ` [PATCH -mm v3 3/8] memcg: mark caches that belong to offline memcgs as dead Vladimir Davydov
2014-06-12 20:38 ` [PATCH -mm v3 4/8] slub: don't fail kmem_cache_shrink if slab placement optimization fails Vladimir Davydov
2014-06-12 20:38 ` [PATCH -mm v3 5/8] slub: make slab_free non-preemptable Vladimir Davydov
2014-06-12 20:38 ` [PATCH -mm v3 6/8] memcg: wait for kfree's to finish before destroying cache Vladimir Davydov
2014-06-12 20:38 ` [PATCH -mm v3 7/8] slub: make dead memcg caches discard free slabs immediately Vladimir Davydov
2014-06-13 16:54 ` Christoph Lameter
2014-06-24 7:50 ` Joonsoo Kim
2014-06-24 8:25 ` Vladimir Davydov
2014-06-24 9:42 ` [PATCH -mm] slub: kmem_cache_shrink: check if partial list is empty under list_lock Vladimir Davydov
2014-06-12 20:38 ` [PATCH -mm v3 8/8] slab: do not keep free objects/slabs on dead memcg caches Vladimir Davydov
2014-06-12 20:41 ` Vladimir Davydov
2014-06-24 7:25 ` Joonsoo Kim
2014-06-24 7:42 ` Vladimir Davydov
2014-06-24 12:28 ` [PATCH -mm] slab: set free_limit for dead caches to 0 Vladimir Davydov
2014-06-24 7:38 ` [PATCH -mm v3 8/8] slab: do not keep free objects/slabs on dead memcg caches Joonsoo Kim
2014-06-24 7:48 ` Vladimir Davydov
2014-06-25 13:45 ` Vladimir Davydov
2014-06-27 6:05 ` Joonsoo Kim [this message]
2014-06-30 15:49 ` Christoph Lameter
2014-07-01 7:46 ` Vladimir Davydov
2014-06-25 14:39 ` [PATCH] slab: document why cache can have no per cpu array on kfree Vladimir Davydov
2014-06-25 16:19 ` Christoph Lameter
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20140627060534.GC9511@js1304-P5Q-DELUXE \
--to=iamjoonsoo.kim@lge.com \
--cc=akpm@linux-foundation.org \
--cc=cl@linux.com \
--cc=hannes@cmpxchg.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@suse.cz \
--cc=penberg@kernel.org \
--cc=rientjes@google.com \
--cc=vdavydov@parallels.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).