From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Boris Zhmurov <bb@kernelpanic.ru>
Cc: Michal Hocko <mhocko@kernel.org>,
Paul Menzel <pmenzel@molgen.mpg.de>,
Donald Buczek <buczek@molgen.mpg.de>,
linux-mm@kvack.org, linux-kernel@vger.kernel.org
Subject: Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and `mem_cgroup_shrink_node`
Date: Thu, 1 Dec 2016 11:39:07 -0800 [thread overview]
Message-ID: <20161201193907.GR3924@linux.vnet.ibm.com> (raw)
In-Reply-To: <e50dcb85-4552-9249-c53e-017fefcaf80b@kernelpanic.ru>
On Thu, Dec 01, 2016 at 09:10:01PM +0300, Boris Zhmurov wrote:
> Michal Hocko 30/11/16 21:25:
>
> >>> Do I get it right that s@cond_resched_rcu_qs@cond_resched@ didn't help?
> >>
> >> I didn't try that. I've tried 4 patches from Paul's linux-rcu tree.
> >> I can try another portion of patches, no problem :)
> >
> > Replacing cond_resched_rcu_qs in shrink_node_memcg by cond_resched would
> > be really helpful to tell whether we are missing a real scheduling point
> > or whether something more serious is going on here.
>
> Well, I can confirm, that replacing cond_resched_rcu_qs in
> shrink_node_memcg by cond_resched also makes dmesg clean from RCU CPU
> stall warnings.
>
> I've attached patch (just modification of Paul's patch), that fixes RCU
> stall messages in situations, when all memory is used by
> couchbase/memcached + fs cache and linux starts to use swap.
>
>
> --
> Boris Zhmurov
> System/Network Administrator
> mailto: bb@kernelpanic.ru
> "wget http://kernelpanic.ru/bb_public_key.pgp -O - | gpg --import"
> --- a/mm/vmscan.c.orig 2016-11-30 21:52:58.314895320 +0300
> +++ b/mm/vmscan.c 2016-11-30 21:53:29.502895320 +0300
> @@ -2352,6 +2352,7 @@
> nr_reclaimed += shrink_list(lru, nr_to_scan,
> lruvec, sc);
> }
> + cond_resched();
> }
>
> if (nr_reclaimed < nr_to_reclaim || scan_adjusted)
Nice!
Just to double-check, could you please also test your patch above with
these two commits from -rcu?
d2db185bfee8 ("rcu: Remove short-term CPU kicking")
f8f127e738e3 ("rcu: Add long-term CPU kicking")
Thanx, Paul
WARNING: multiple messages have this Message-ID (diff)
From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Boris Zhmurov <bb@kernelpanic.ru>
Cc: Michal Hocko <mhocko@kernel.org>,
Paul Menzel <pmenzel@molgen.mpg.de>,
Donald Buczek <buczek@molgen.mpg.de>,
linux-mm@kvack.org, linux-kernel@vger.kernel.org
Subject: Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and `mem_cgroup_shrink_node`
Date: Thu, 1 Dec 2016 11:39:07 -0800 [thread overview]
Message-ID: <20161201193907.GR3924@linux.vnet.ibm.com> (raw)
In-Reply-To: <e50dcb85-4552-9249-c53e-017fefcaf80b@kernelpanic.ru>
On Thu, Dec 01, 2016 at 09:10:01PM +0300, Boris Zhmurov wrote:
> Michal Hocko 30/11/16 21:25:
>
> >>> Do I get it right that s@cond_resched_rcu_qs@cond_resched@ didn't help?
> >>
> >> I didn't try that. I've tried 4 patches from Paul's linux-rcu tree.
> >> I can try another portion of patches, no problem :)
> >
> > Replacing cond_resched_rcu_qs in shrink_node_memcg by cond_resched would
> > be really helpful to tell whether we are missing a real scheduling point
> > or whether something more serious is going on here.
>
> Well, I can confirm, that replacing cond_resched_rcu_qs in
> shrink_node_memcg by cond_resched also makes dmesg clean from RCU CPU
> stall warnings.
>
> I've attached patch (just modification of Paul's patch), that fixes RCU
> stall messages in situations, when all memory is used by
> couchbase/memcached + fs cache and linux starts to use swap.
>
>
> --
> Boris Zhmurov
> System/Network Administrator
> mailto: bb@kernelpanic.ru
> "wget http://kernelpanic.ru/bb_public_key.pgp -O - | gpg --import"
> --- a/mm/vmscan.c.orig 2016-11-30 21:52:58.314895320 +0300
> +++ b/mm/vmscan.c 2016-11-30 21:53:29.502895320 +0300
> @@ -2352,6 +2352,7 @@
> nr_reclaimed += shrink_list(lru, nr_to_scan,
> lruvec, sc);
> }
> + cond_resched();
> }
>
> if (nr_reclaimed < nr_to_reclaim || scan_adjusted)
Nice!
Just to double-check, could you please also test your patch above with
these two commits from -rcu?
d2db185bfee8 ("rcu: Remove short-term CPU kicking")
f8f127e738e3 ("rcu: Add long-term CPU kicking")
Thanx, Paul
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2016-12-01 19:39 UTC|newest]
Thread overview: 94+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <d6981bac-8e97-b482-98c0-40949db03ca3@kernelpanic.ru>
[not found] ` <20161124133019.GE3612@linux.vnet.ibm.com>
[not found] ` <de88a72a-f861-b51f-9fb3-4265378702f1@kernelpanic.ru>
[not found] ` <20161125212000.GI31360@linux.vnet.ibm.com>
[not found] ` <20161128095825.GI14788@dhcp22.suse.cz>
[not found] ` <20161128105425.GY31360@linux.vnet.ibm.com>
[not found] ` <3a4242cb-0198-0a3b-97ae-536fb5ff83ec@kernelpanic.ru>
[not found] ` <20161128143435.GC3924@linux.vnet.ibm.com>
2016-11-28 14:40 ` INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and `mem_cgroup_shrink_node` Boris Zhmurov
2016-11-28 15:05 ` Paul E. McKenney
2016-11-28 19:16 ` Boris Zhmurov
2016-11-29 18:59 ` Paul E. McKenney
2016-11-30 17:41 ` Boris Zhmurov
2016-11-30 17:48 ` Michal Hocko
2016-11-30 18:12 ` Boris Zhmurov
2016-11-30 18:25 ` Michal Hocko
2016-11-30 18:26 ` Boris Zhmurov
2016-12-01 18:10 ` Boris Zhmurov
2016-12-01 19:39 ` Paul E. McKenney [this message]
2016-12-01 19:39 ` Paul E. McKenney
2016-12-02 9:37 ` Michal Hocko
2016-12-02 9:37 ` Michal Hocko
2016-12-02 13:52 ` Paul E. McKenney
2016-12-02 13:52 ` Paul E. McKenney
2016-12-02 16:39 ` Boris Zhmurov
2016-12-02 16:39 ` Boris Zhmurov
2016-12-02 16:44 ` Paul E. McKenney
2016-12-02 16:44 ` Paul E. McKenney
2016-12-02 17:02 ` Michal Hocko
2016-12-02 17:02 ` Michal Hocko
2016-12-02 17:15 ` Paul E. McKenney
2016-12-02 17:15 ` Paul E. McKenney
2016-11-30 19:42 ` Paul E. McKenney
[not found] <24c226a5-1a4a-173e-8b4e-5107a2baac04@molgen.mpg.de>
2016-11-08 12:22 ` Paul Menzel
2016-11-08 17:03 ` Paul E. McKenney
2016-11-08 17:38 ` Paul Menzel
2016-11-08 18:39 ` Paul E. McKenney
2016-11-16 17:01 ` Paul Menzel
2016-11-16 17:30 ` Paul E. McKenney
2016-11-21 13:41 ` Michal Hocko
2016-11-21 14:01 ` Paul E. McKenney
2016-11-21 14:18 ` Michal Hocko
2016-11-21 14:29 ` Paul E. McKenney
2016-11-21 15:35 ` Donald Buczek
2016-11-24 10:15 ` Michal Hocko
2016-11-24 18:50 ` Donald Buczek
2016-11-27 9:37 ` Paul Menzel
2016-11-27 5:32 ` Christopher S. Aker
2016-11-27 9:19 ` Donald Buczek
2016-11-28 11:04 ` Michal Hocko
2016-11-28 12:26 ` Paul Menzel
2016-11-28 12:26 ` Paul Menzel
2016-11-30 10:28 ` Donald Buczek
2016-11-30 10:28 ` Donald Buczek
2016-11-30 11:09 ` Michal Hocko
2016-11-30 11:09 ` Michal Hocko
2016-11-30 11:43 ` Donald Buczek
2016-11-30 11:43 ` Donald Buczek
2016-12-02 9:14 ` Donald Buczek
2016-12-02 9:14 ` Donald Buczek
2016-12-06 8:32 ` Donald Buczek
2016-12-06 8:32 ` Donald Buczek
2016-11-30 11:53 ` Paul E. McKenney
2016-11-30 11:53 ` Paul E. McKenney
2016-11-30 11:54 ` Paul E. McKenney
2016-11-30 11:54 ` Paul E. McKenney
2016-11-30 12:31 ` Paul Menzel
2016-11-30 12:31 ` Paul Menzel
2016-11-30 14:31 ` Paul E. McKenney
2016-11-30 14:31 ` Paul E. McKenney
2016-11-30 13:19 ` Michal Hocko
2016-11-30 13:19 ` Michal Hocko
2016-11-30 14:29 ` Paul E. McKenney
2016-11-30 14:29 ` Paul E. McKenney
2016-11-30 16:38 ` Peter Zijlstra
2016-11-30 16:38 ` Peter Zijlstra
2016-11-30 17:02 ` Paul E. McKenney
2016-11-30 17:02 ` Paul E. McKenney
2016-11-30 17:05 ` Michal Hocko
2016-11-30 17:05 ` Michal Hocko
2016-11-30 17:23 ` Paul E. McKenney
2016-11-30 17:23 ` Paul E. McKenney
2016-11-30 17:34 ` Michal Hocko
2016-11-30 17:34 ` Michal Hocko
2016-11-30 17:50 ` Peter Zijlstra
2016-11-30 17:50 ` Peter Zijlstra
2016-11-30 19:40 ` Paul E. McKenney
2016-11-30 19:40 ` Paul E. McKenney
2016-12-01 5:30 ` Peter Zijlstra
2016-12-01 5:30 ` Peter Zijlstra
2016-12-01 12:40 ` Paul E. McKenney
2016-12-01 12:40 ` Paul E. McKenney
2016-12-01 16:36 ` Peter Zijlstra
2016-12-01 16:36 ` Peter Zijlstra
2016-12-01 16:59 ` Paul E. McKenney
2016-12-01 16:59 ` Paul E. McKenney
2016-12-01 18:09 ` Peter Zijlstra
2016-12-01 18:09 ` Peter Zijlstra
2016-12-01 18:42 ` Paul E. McKenney
2016-12-01 18:42 ` Paul E. McKenney
2016-12-01 18:49 ` Peter Zijlstra
2016-12-01 18:49 ` Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20161201193907.GR3924@linux.vnet.ibm.com \
--to=paulmck@linux.vnet.ibm.com \
--cc=bb@kernelpanic.ru \
--cc=buczek@molgen.mpg.de \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@kernel.org \
--cc=pmenzel@molgen.mpg.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.