From: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
To: hughd@google.com
Cc: mhocko@kernel.org, akpm@linux-foundation.org, mgorman@suse.de,
riel@redhat.com, hannes@cmpxchg.org, vbabka@suse.cz,
linux-mm@kvack.org, linux-kernel@vger.kernel.org,
mhocko@suse.com
Subject: Re: [PATCH] mm, vmscan: do not loop on too_many_isolated for ever
Date: Mon, 24 Jul 2017 20:12:13 +0900 [thread overview]
Message-ID: <201707242012.CJJ06237.tVFQSOFFJMOHOL@I-love.SAKURA.ne.jp> (raw)
In-Reply-To: <alpine.LSU.2.11.1707232339430.2154@eggly.anvils>
Hugh Dickins wrote:
> On Thu, 20 Jul 2017, Tetsuo Handa wrote:
> > Hugh Dickins wrote:
> > > You probably won't welcome getting into alternatives at this late stage;
> > > but after hacking around it one way or another because of its pointless
> > > lockups, I lost patience with that too_many_isolated() loop a few months
> > > back (on realizing the enormous number of pages that may be isolated via
> > > migrate_pages(2)), and we've been running nicely since with something like:
> > >
> > > bool got_mutex = false;
> > >
> > > if (unlikely(too_many_isolated(pgdat, file, sc))) {
> > > if (mutex_lock_killable(&pgdat->too_many_isolated))
> > > return SWAP_CLUSTER_MAX;
> > > got_mutex = true;
> > > }
> > > ...
> > > if (got_mutex)
> > > mutex_unlock(&pgdat->too_many_isolated);
> > >
> > > Using a mutex to provide the intended throttling, without an infinite
> > > loop or an arbitrary delay; and without having to worry (as we often did)
> > > about whether those numbers in too_many_isolated() are really appropriate.
> > > No premature OOMs complained of yet.
> >
> > Roughly speaking, there is a moment where shrink_inactive_list() acts
> > like below.
> >
> > bool got_mutex = false;
> >
> > if (!current_is_kswapd()) {
> > if (mutex_lock_killable(&pgdat->too_many_isolated))
> > return SWAP_CLUSTER_MAX;
> > got_mutex = true;
> > }
> >
> > // kswapd is blocked here waiting for !current_is_kswapd().
>
> That would be a shame, for kswapd to wait for !current_is_kswapd()!
Yes, but current code (not about your patch) does allow kswapd to wait
for memory allocations of !current_is_kswapd() thread to complete.
>
> But seriously, I think I understand what you mean by that, you're
> thinking that kswapd would be waiting on some other task to clear
> the too_many_isolated() condition?
Yes.
>
> No, it does not work that way: kswapd (never seeing too_many_isolated()
> because that always says false when current_is_kswapd()) never tries to
> take the pgdat->too_many_isolated mutex itself: it does not wait there
> at all, although other tasks may be waiting there at the time.
I know. I wrote behavior of your patch if my guess (your "..." part
corresponds to kswapd doing writepage) is correct.
>
> Perhaps my naming the mutex "too_many_isolated", same as the function,
> is actually confusing, when I had intended it to be helpful.
Not confusing at all. It is helpful.
I just wanted to confirm what comes in your "..." part.
>
> >
> > if (got_mutex)
> > mutex_unlock(&pgdat->too_many_isolated);
> >
> > >
> > > But that was on a different kernel, and there I did have to make sure
> > > that PF_MEMALLOC always prevented us from nesting: I'm not certain of
> > > that in the current kernel (but do remember Johannes changing the memcg
> > > end to make it use PF_MEMALLOC too). I offer the preview above, to see
> > > if you're interested in that alternative: if you are, then I'll go ahead
> > > and make it into an actual patch against v4.13-rc.
> >
> > I don't know what your actual patch looks like, but the problem is that
> > pgdat->too_many_isolated waits for kswapd while kswapd waits for
> > pgdat->too_many_isolated; nobody can unlock pgdat->too_many_isolated if
> > once we hit it.
>
> Not so (and we'd hardly be finding it a useful patch if that were so).
Current code allows kswapd to wait for memory allocation of !current_is_kswapd()
threads, and thus !current_is_kswapd() threads wait for current_is_kswapd() threads
while current_is_kswapd() threads wait for !current_is_kswapd() threads; nobody can
make too_many_isolated() false if once we hit it. Hence, this patch is proposed.
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2017-07-24 11:12 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-07-10 7:48 [PATCH] mm, vmscan: do not loop on too_many_isolated for ever Michal Hocko
2017-07-10 13:16 ` Vlastimil Babka
2017-07-10 13:58 ` Rik van Riel
2017-07-10 16:58 ` Johannes Weiner
2017-07-10 17:09 ` Michal Hocko
2017-07-19 22:20 ` Andrew Morton
2017-07-20 6:56 ` Michal Hocko
2017-07-21 23:01 ` Andrew Morton
2017-07-24 6:50 ` Michal Hocko
2017-07-20 1:54 ` Hugh Dickins
2017-07-20 10:44 ` Tetsuo Handa
2017-07-24 7:01 ` Hugh Dickins
2017-07-24 11:12 ` Tetsuo Handa [this message]
2017-07-20 13:22 ` Michal Hocko
2017-07-24 7:03 ` Hugh Dickins
-- strict thread matches above, loose matches on Subject: below --
2017-03-07 13:30 Michal Hocko
2017-03-07 19:52 ` Rik van Riel
2017-03-08 9:21 ` Michal Hocko
2017-03-08 15:54 ` Rik van Riel
2017-03-09 9:12 ` Michal Hocko
2017-03-09 14:16 ` Rik van Riel
2017-03-09 14:59 ` Michal Hocko
2017-03-09 18:05 ` Johannes Weiner
2017-03-09 22:18 ` Rik van Riel
2017-03-10 10:27 ` Michal Hocko
2017-03-10 10:20 ` Michal Hocko
2017-03-10 11:44 ` Tetsuo Handa
2017-03-21 10:37 ` Tetsuo Handa
2017-04-23 10:24 ` Tetsuo Handa
2017-04-24 12:39 ` Stanislaw Gruszka
2017-04-24 13:06 ` Tetsuo Handa
2017-04-25 6:33 ` Stanislaw Gruszka
2017-06-30 0:14 ` Tetsuo Handa
2017-06-30 13:32 ` Michal Hocko
2017-06-30 15:59 ` Tetsuo Handa
2017-06-30 16:19 ` Michal Hocko
2017-07-01 11:43 ` Tetsuo Handa
2017-07-05 8:19 ` Michal Hocko
2017-07-05 8:20 ` Michal Hocko
2017-07-06 10:48 ` Tetsuo Handa
2017-03-09 14:31 ` Mel Gorman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=201707242012.CJJ06237.tVFQSOFFJMOHOL@I-love.SAKURA.ne.jp \
--to=penguin-kernel@i-love.sakura.ne.jp \
--cc=akpm@linux-foundation.org \
--cc=hannes@cmpxchg.org \
--cc=hughd@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@suse.de \
--cc=mhocko@kernel.org \
--cc=mhocko@suse.com \
--cc=riel@redhat.com \
--cc=vbabka@suse.cz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).