From: Hugh Dickins <hughd@google.com>
To: Michal Hocko <mhocko@kernel.org>
Cc: Alex Shi <alex.shi@linux.alibaba.com>,
Cgroups <cgroups@vger.kernel.org>,
LKML <linux-kernel@vger.kernel.org>,
Linux MM <linux-mm@kvack.org>,
Andrew Morton <akpm@linux-foundation.org>,
Mel Gorman <mgorman@techsingularity.net>,
Tejun Heo <tj@kernel.org>, Hugh Dickins <hughd@google.com>,
Shakeel Butt <shakeelb@google.com>, Yu Zhao <yuzhao@google.com>,
Daniel Jordan <daniel.m.jordan@oracle.com>
Subject: Re: [PATCH 00/14] per memcg lru_lock
Date: Tue, 20 Aug 2019 11:24:44 -0700 (PDT) [thread overview]
Message-ID: <alpine.LSU.2.11.1908201038260.1286@eggly.anvils> (raw)
In-Reply-To: <CALvZod7-dL90jwd2pywpaD8NfUByVU9Y809+RfvJABGdRASYUg@mail.gmail.com>
[-- Attachment #1: Type: TEXT/PLAIN, Size: 2042 bytes --]
On Tue, 20 Aug 2019, Shakeel Butt wrote:
> On Tue, Aug 20, 2019 at 3:45 AM Michal Hocko <mhocko@kernel.org> wrote:
> > On Tue 20-08-19 17:48:23, Alex Shi wrote:
> > > This patchset move lru_lock into lruvec, give a lru_lock for each of
> > > lruvec, thus bring a lru_lock for each of memcg.
> > >
> > > Per memcg lru_lock would ease the lru_lock contention a lot in
> > > this patch series.
> > >
> > > In some data center, containers are used widely to deploy different kind
> > > of services, then multiple memcgs share per node pgdat->lru_lock which
> > > cause heavy lock contentions when doing lru operation.
> >
> > Having some real world workloads numbers would be more than useful
> > for a non trivial change like this. I believe googlers have tried
> > something like this in the past but then didn't have really a good
> > example of workloads that benefit. I might misremember though. Cc Hugh.
> >
>
> We, at Google, have been using per-memcg lru locks for more than 7
> years. Per-memcg lru locks are really beneficial for providing
> performance isolation if there are multiple distinct jobs/memcgs
> running on large machines. We are planning to upstream our internal
> implementation. I will let Hugh comment on that.
Thanks for the Cc Michal. As Shakeel says, Google prodkernel has been
using our per-memcg lru locks for 7 years or so. Yes, we did not come
up with supporting performance data at the time of posting, nor since:
I see Alex has done much better on that (though I haven't even glanced
to see if +s are persuasive).
https://lkml.org/lkml/2012/2/20/434
was how ours was back then; some parts of that went in, then attached
lrulock417.tar is how it was the last time I rebased, to v4.17.
I'll set aside what I'm doing, and switch to rebasing ours to v5.3-rc
and/or mmotm. Then compare with what Alex has, to see if there's any
good reason to prefer one to the other: if no good reason to prefer ours,
I doubt we shall bother to repost, but just use it as basis for helping
to review or improve Alex's.
Hugh
[-- Attachment #2: per-memcg lru_lock on v4.17 --]
[-- Type: APPLICATION/x-tar, Size: 112640 bytes --]
next prev parent reply other threads:[~2019-08-20 18:25 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-08-20 9:48 [PATCH 00/14] per memcg lru_lock Alex Shi
2019-08-20 9:48 ` [PATCH 01/14] mm/lru: move pgdat lru_lock into lruvec Alex Shi
2019-08-20 13:40 ` Matthew Wilcox
2019-08-20 14:11 ` Alex Shi
2019-08-20 9:48 ` [PATCH 02/14] lru/memcg: move the lruvec->pgdat sync out lru_lock Alex Shi
2019-08-20 9:48 ` [PATCH 03/14] lru/memcg: using per lruvec lock in un/lock_page_lru Alex Shi
2019-08-26 8:30 ` Konstantin Khlebnikov
2019-08-26 14:16 ` Alex Shi
2019-08-20 9:48 ` [PATCH 04/14] lru/compaction: use per lruvec lock in isolate_migratepages_block Alex Shi
2019-08-20 9:48 ` [PATCH 05/14] lru/huge_page: use per lruvec lock in __split_huge_page Alex Shi
2019-08-20 9:48 ` [PATCH 06/14] lru/mlock: using per lruvec lock in munlock Alex Shi
2019-08-20 9:48 ` [PATCH 07/14] lru/swap: using per lruvec lock in page_cache_release Alex Shi
2019-08-20 9:48 ` [PATCH 08/14] lru/swap: uer lruvec lock in activate_page Alex Shi
2019-08-20 9:48 ` [PATCH 09/14] lru/swap: uer per lruvec lock in pagevec_lru_move_fn Alex Shi
2019-08-20 9:48 ` [PATCH 10/14] lru/swap: use per lruvec lock in release_pages Alex Shi
2019-08-20 9:48 ` [PATCH 11/14] lru/vmscan: using per lruvec lock in lists shrinking Alex Shi
2019-08-20 9:48 ` [PATCH 12/14] lru/vmscan: use pre lruvec lock in check_move_unevictable_pages Alex Shi
2019-08-20 9:48 ` [PATCH 13/14] lru/vmscan: using per lruvec lru_lock in get_scan_count Alex Shi
2019-08-20 9:48 ` [PATCH 14/14] mm/lru: fix the comments of lru_lock Alex Shi
2019-08-20 14:00 ` Matthew Wilcox
2019-08-20 14:21 ` Alex Shi
2019-08-20 10:45 ` [PATCH 00/14] per memcg lru_lock Michal Hocko
2019-08-20 16:48 ` Shakeel Butt
2019-08-20 18:24 ` Hugh Dickins [this message]
2019-08-21 1:21 ` Alex Shi
2019-08-21 2:00 ` Alex Shi
2019-08-24 1:59 ` Hugh Dickins
2019-08-26 14:35 ` Alex Shi
2019-08-21 18:00 ` Daniel Jordan
2019-08-22 11:56 ` Alex Shi
2019-08-22 15:20 ` Daniel Jordan
2019-08-26 8:39 ` Konstantin Khlebnikov
2019-08-26 14:22 ` Alex Shi
2019-08-26 14:25 ` Alex Shi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=alpine.LSU.2.11.1908201038260.1286@eggly.anvils \
--to=hughd@google.com \
--cc=akpm@linux-foundation.org \
--cc=alex.shi@linux.alibaba.com \
--cc=cgroups@vger.kernel.org \
--cc=daniel.m.jordan@oracle.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@techsingularity.net \
--cc=mhocko@kernel.org \
--cc=shakeelb@google.com \
--cc=tj@kernel.org \
--cc=yuzhao@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).